Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ASP.NET Core Module fails to load the runtime when the Null Service is stopped #46544

Closed
1 task done
mertozturk80 opened this issue Feb 9, 2023 · 6 comments
Closed
1 task done
Assignees
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions Docs This issue tracks updating documentation feature-iis Includes: IIS, ANCM net8_docathon This issue is part of the documentation push in Fall 2023.
Milestone

Comments

@mertozturk80
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

Can aspnetcorev2.dll be dependent on null device, especially when it makes Standard stream redirection?
We have discovered an issue for a customer that if you stop the Null Service (and restart the worker), the startup of ANCM is failing. Ie:

C:\Windows\System32>sc stop null
..

With in-process hosting, we have the following callstack in the crash dump, of IIS Worker:

0:008> !gle
LastErrorValue: 0x00000002 = ERROR_FILE_NOT_FOUND
LastStatusValue: 0xc0000034 = STATUS_OBJECT_NAME_NOT_FOUND

Failure Stack 
0:008> kL

Child-SP RetAddr Call Site

00 000000e7389fea70 00007ffb3763c8b1 ucrtbase!_invoke_watson+0x18
01 000000e7389feaa0 00007ffb3763c989 ucrtbase!_invalid_parameter+0x81
02 000000e7389feae0 00007ffb37619936 ucrtbase!_invalid_parameter_noinfo+0x9
03 000000e7389feb20 00007ffafed873c8 ucrtbase!_fileno+0x35c16
04 000000e7389feb50 00007ffafed7e839 aspnetcorev2!StdWrapper::StartRedirection+0x48
05 000000e7389feba0 00007ffafed7eb09 aspnetcorev2!StandardStreamRedirection::Start+0x1a9
06 000000e7389fec20 00007ffafed7e303 aspnetcorev2!StandardStreamRedirection::TryStartRedirection+0x9
07 000000e7389fec50 00007ffafed6d243 aspnetcorev2!StandardStreamRedirection::StandardStreamRedirection+0x43
08 000000e7389fec80 00007ffafed6ba8d aspnetcorev2!HandlerResolver::FindNativeAssemblyFromHostfxr+0x393
09 000000e7389fef80 00007ffafed6c89f aspnetcorev2!HandlerResolver::LoadRequestHandlerAssembly+0x39d
0a 000000e7389ff170 00007ffafed63552 aspnetcorev2!HandlerResolver::GetApplicationFactory+0x6bf
0b 000000e7389ff2c0 00007ffafed62b18 aspnetcorev2!APPLICATION_INFO::TryCreateApplication+0x322
0c 000000e7389ff410 00007ffafed627f7 aspnetcorev2!APPLICATION_INFO::CreateApplication+0x268
0d 000000e7389ff610 00007ffafed6f670 aspnetcorev2!APPLICATION_INFO::CreateHandler+0x157
0e 000000e7389ff680 00007ffb076cd3f9 aspnetcorev2!ASPNET_CORE_PROXY_MODULE::OnExecuteRequestHandler+0x110
..

I suspect the following section of the code, where we are creating a Nul file -->

"IIS\AspNetCoreModuleV2\CommonLib\StdWrapper.cpp @ 51 "

HRESULT
StdWrapper::StartRedirection()
{
..
// Open a null file such that undoing the redirection succeeds and _dup2 works.
// m_previousFileDescriptor will be used for restoring stdout/stderr
if (_fileno(m_stdStream) == -2)
{
freopen_s((FILE**)&m_stdStream, "nul", "w", m_stdStream);
m_previousFileDescriptor = _dup(_fileno(m_stdStream));
}
..

By Default the Null service is in running mode, and its a sort of hidden service. However i can see that, there are some people who are experiencing this issue, and underlying reason remains as a mystery. See the StackOverflow article:

https://stackoverflow.com/questions/72591225/asp-net-core-3-1-iis-worker-crashes-on-startup

Even if we decide not to fix this, we should be documenting it, here or somewhere.

Expected Behavior

aspnetcorev2.dll to handle the abcense of the Null service dependency. Or we should add this as a requirenment in the documentation.

Steps To Reproduce

#1 - Stop the Null Service on a Windows system

C:\Windows\System32>sc stop null
..

#2 - Restart the IIS Worker Process hosting an in-proc application.
#3- Site will start to fail with a Worker process (w3wp.exe) crash.

Exceptions (if any)

ExceptionAddress: 00007ffb3763c9e8 (ucrtbase!_invoke_watson+0x0000000000000018)
ExceptionCode: c0000409 (Security check failure or stack buffer overrun)
ExceptionFlags: 00000001
NumberParameters: 1
Parameter[0]: 0000000000000005
Subcode: 0x5 FAST_FAIL_INVALID_ARG

.NET Version

Version independent

Anything else?

No response

@Tratcher Tratcher added the feature-iis Includes: IIS, ANCM label Feb 9, 2023
@adityamandaleeka adityamandaleeka added this to the 8.0-preview2 milestone Feb 13, 2023
@adityamandaleeka
Copy link
Member

@mertozturk80 Thanks for reporting this. Your analysis is correct, and that is indeed where we are failing.

I'll need to think about this a bit to decide what we should do here...

@adityamandaleeka
Copy link
Member

@mertozturk80 I'm really curious, is there a reason you are disabling the null service in the first place? I looked into this a bit and it seems like lots of things encounter errors when the null service is stopped. I don't think there's an expectation that ASP.NET Core will still work when core parts of the OS/environment are disabled.

@adityamandaleeka adityamandaleeka added the Needs: Author Feedback The author of this issue needs to respond in order for us to continue investigating this issue. label Feb 25, 2023
@ghost
Copy link

ghost commented Feb 25, 2023

Hi @mertozturk80. We have added the "Needs: Author Feedback" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

@ghost
Copy link

ghost commented Mar 1, 2023

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment. If it is closed, feel free to comment when you are able to provide the additional information and we will re-investigate.

See our Issue Management Policies for more information.

@ghost ghost closed this as completed Mar 6, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Apr 5, 2023
@halter73 halter73 added the Docs This issue tracks updating documentation label Apr 26, 2023
@halter73 halter73 removed the Needs: Author Feedback The author of this issue needs to respond in order for us to continue investigating this issue. label Apr 26, 2023
@halter73 halter73 removed this from the 8.0-preview2 milestone Apr 26, 2023
@halter73 halter73 reopened this Apr 26, 2023
@halter73
Copy link
Member

I'm really curious, is there a reason you are disabling the null service in the first place?

Without a good reason to disable the service in the first place, I figure documentation should be sufficient. I'm reopening this because a request on an internal email thread to document this after someone else ran into the issue. We could probably add something to the troubleshooting guide.

@amcasey amcasey added area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions and removed area-runtime labels Jun 2, 2023
@adityamandaleeka adityamandaleeka added the net8_docathon This issue is part of the documentation push in Fall 2023. label Aug 25, 2023
@BrennanConroy
Copy link
Member

Now documented.

@BrennanConroy BrennanConroy added this to the 8.0 milestone Sep 8, 2023
@wtgodbe wtgodbe modified the milestones: 8.0, 8.0.0 Oct 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions Docs This issue tracks updating documentation feature-iis Includes: IIS, ANCM net8_docathon This issue is part of the documentation push in Fall 2023.
Projects
None yet
Development

No branches or pull requests

7 participants