Tennant gone missing intermittently when deployed on Azure Web App

[larremp]

Background:

1. I have 2 sites in Azure. The first, site A, is on dev v1.0.0.0-betaB-69950 (rough guess). Site A has 3 tenants with custom domains and running with no issues.
2. The second site, site B, is on dev v1.0.0-beta3-70374. I was in the process of migrating the tenants on Site A to site B when saw the issues outlined below under the Error section.
3. Site A is still working with no issues. I had to put the tenants back to site A since site B had issues.
4. Please look at the Scenario section below go know what OC is running on Azure.

Error:

1. The tenants on OC started to get missing when 2nd through 4th tenant were added.
2. The first tenant was working fine, even after adding a custom domain.
3. Adding the 2nd through 4th tenants was still ok.
4. Adding the custom domain on the 2nd through 4th tenants seems to trigger the issue.( May or may not be true).
5. Error code from the log
   2019-01-05 18:47:06.7974|None|0HLJJ7OO04QNO:00000001||OrchardCore.Environment.Shell.ShellHost|ERROR|A tenant could not be started 't4basicstostart' System.ObjectDisposedException: Cannot access a disposed object.
   Object name: 'IServiceProvider'.
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ThrowHelper.ThrowObjectDisposedException()
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at OrchardCore.Mvc.ShellViewFeatureProvider..ctor(IHttpContextAccessor httpContextAccessor) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore.Mvc.Core\ShellViewFeatureProvider.cs:line 27
   at OrchardCore.Mvc.Startup.AddModularFrameworkParts(IServiceProvider services, ApplicationPartManager manager) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore.Mvc.Core\Startup.cs:line 69
   at OrchardCore.Mvc.Startup.ConfigureServices(IServiceCollection services) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore.Mvc.Core\Startup.cs:line 50
   at OrchardCore.Environment.Shell.Builders.ShellContainerFactory.CreateContainer(ShellSettings settings, ShellBlueprint blueprint) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore\Environment\Shell\Builders\ShellContainerFactory.cs:line 96
   at OrchardCore.Environment.Shell.Builders.ShellContextFactory.CreateDescribedContextAsync(ShellSettings settings, ShellDescriptor shellDescriptor) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore\Environment\Shell\Builders\ShellContextFactory.cs:line 75
   at OrchardCore.Environment.Shell.Builders.ShellContextFactory.OrchardCore.Environment.Shell.Builders.IShellContextFactory.CreateShellContextAsync(ShellSettings settings) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore\Environment\Shell\Builders\ShellContextFactory.cs:line 49
   at OrchardCore.Environment.Shell.ShellHost.GetOrCreateShellContextAsync(ShellSettings settings) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore\Environment\Shell\ShellHost.cs:line 89
   at OrchardCore.Environment.Shell.ShellHost.b__15_0(ShellSettings settings) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore\Environment\Shell\ShellHost.cs:line 184 at Microsoft.Extensions.DependencyInjection.ServiceLookup.ThrowHelper.ThrowObjectDisposedException()
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at OrchardCore.Mvc.ShellViewFeatureProvider..ctor(IHttpContextAccessor httpContextAccessor) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore.Mvc.Core\ShellViewFeatureProvider.cs:line 27
   at OrchardCore.Mvc.Startup.AddModularFrameworkParts(IServiceProvider services, ApplicationPartManager manager) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore.Mvc.Core\Startup.cs:line 69
   at OrchardCore.Mvc.Startup.ConfigureServices(IServiceCollection services) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore.Mvc.Core\Startup.cs:line 50
   at OrchardCore.Environment.Shell.Builders.ShellContainerFactory.CreateContainer(ShellSettings settings, ShellBlueprint blueprint) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore\Environment\Shell\Builders\ShellContainerFactory.cs:line 96
   at OrchardCore.Environment.Shell.Builders.ShellContextFactory.CreateDescribedContextAsync(ShellSettings settings, ShellDescriptor shellDescriptor) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore\Environment\Shell\Builders\ShellContextFactory.cs:line 75
   at OrchardCore.Environment.Shell.Builders.ShellContextFactory.OrchardCore.Environment.Shell.Builders.IShellContextFactory.CreateShellContextAsync(ShellSettings settings) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore\Environment\Shell\Builders\ShellContextFactory.cs:line 49
   at OrchardCore.Environment.Shell.ShellHost.GetOrCreateShellContextAsync(ShellSettings settings) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore\Environment\Shell\ShellHost.cs:line 89
   at OrchardCore.Environment.Shell.ShellHost.b__15_0(ShellSettings settings) in C:\dev\oc\repos\busyaprons\src\OrchardCore\OrchardCore\Environment\Shell\ShellHost.cs:line 184

Scenario

1. Deployed on Azure
2. Location: Central US
3. App Service Plan: Both on D1 and B1 pricing tiers
4. SQL Server - Azure SQL Database

[jtkech]

Okay, i was able to repro.

2019-01-07 00:21:40.0231|None|0HLJK5ML28GM3:00000001||OrchardCore.Environment.Shell.ShellHost|ERROR|A tenant could not be started 'Default' System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'IServiceProvider'.
at

And so on.

Regarding where it is thrown, i know how to fix it. But first i want to well understand what happens.

[sebastienros]

I will merge the fix. Do we know since when this is happening?

[larremp]

My rough estimate is this started happening only in December when I tried to add tenants on my site B. My site A is working fine with multiple tenants, but running on older versions of dev code.

I'm not sure if this answers your question. Please tell me if you need further details. Thanks.

[sebastienros]

It was more a question for @jtkech as he found the root cause.

[jtkech]

Since 9a836d2 related to one of my pr #2940 where i introduced ShellViewFeatureProvider which uses RequestServices to resolve application singletons in its ctor. An easy fix was to resolve them later but better to fix the underlying problem highlighted by this issue.