Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSI_SECRET and MSI_ENDPOINT are not populated after enabling MSI #2038

Closed
martinpeck opened this issue Oct 16, 2017 · 8 comments
Closed

MSI_SECRET and MSI_ENDPOINT are not populated after enabling MSI #2038

martinpeck opened this issue Oct 16, 2017 · 8 comments

Comments

@martinpeck
Copy link
Member

After enabling the Managed Service Identity for an Function App, the MSI_SECRET and MSI_ENDPOINT environment variables are supposed to be populated. However, in my experience, this can take a long time to happen. I've tried:

  • killing the processes (via Kudu Process Explorer)
  • restarting the application
  • applying additional application settings (that are then seen as env variables)
  • deploying new code

None of these actions reliably causes the MSI_SECRET or MSI_ENDPOINT environment variable to be populated. In fact, in some situations where these environment variables have been set up correctly, some of the above actions (in particular, redeploying code) can cause them to disappear.

Investigative information

Details from a run where this issue was seen:

2017-10-16T21:33:14.186 Function started (Id=61e648a4-291f-452c-8c55-827da337748b)
2017-10-16T21:33:14.233 JavaScript HTTP trigger function processed a request.
2017-10-16T21:33:14.233 Function completed (Success, Id=61e648a4-291f-452c-8c55-827da337748b, Duration=36ms)

This is a Javascript HttpTrigger function, running in West Europe.

Repro steps

  1. Create a new Function Application
  2. Create a new Function (JavaScript in this case)
  3. Apply the following implementation...
module.exports = function (context, req) {
    context.log('JavaScript HTTP trigger function processed a request.');

    context.res = process.env["MSI_ENDPOINT"] || "value not set";

    context.done();
};
  1. Save the function defintion
  2. Open the Platform Features for the function app
  3. Select "Managed Service Identity"
  4. Switch MSI ON
  5. Wait for the application to register with Active Directory

EXPECTED

  • running the function now should return the value of MSI_ENDPOINT
  • opening the Console for the function app, and typing env should show the MSI_ENDPOINT and MSI_SECRET environment variables
  • opening the Kudu console and typing env should also show these values
  • viewing the process properties in Kudu's process explorer should show the environment variables

ACTUAL

  • "value not set" is returned
  • none of the techniques described in the EXPECTED show the environment variabes
  • after a period of time, or after killing processes, the environment variables may appear
  • changing the environment variable in the function to be something defined in application settings results in the code working fine
@martinpeck
Copy link
Member Author

It appears that only by...

  1. leaving the function app over night (12 hours)
  2. using Kudu process explorer to kill the process

... can I get this to work as expected. The application I used for repro'ing this issue now has these env variables set up as expected.

What didn't work was...

  1. Leave for 12 hours
  2. Try hitting the function endpoint (returned "value not set")
  3. Try restarting the function app from the portal (returned "value not set")

@davidebbo
Copy link
Contributor

davidebbo commented Oct 17, 2017

This is a known issue affecting a few scale units, mostly in West Europe. It will be fixed within a couple weeks, but until then you can work around by setting WEBSITE_USE_PLACEHOLDER=0 in your Function App Settings.

@martinpeck
Copy link
Member Author

Aha....interesting. Does this mean that other regions are less likely to see this problem? I'll try that setting too. Thanks @davidebbo

@martinpeck
Copy link
Member Author

you can work around by setting WEBSITE_USE_PLACEHOLDER=0 in your Function App Settings.

I've confirmed that this works. I needed to restart (using the portal) the Function app, but right after that the MSI_* environment variables were available.

@paulbatum
Copy link
Member

Closing this as the fix has been checked in to app service. As David mentioned above, deployment is probably still a couple of weeks away.

@martinpeck
Copy link
Member Author

Thanks for resolving this. Will there be any update here (or elsewhere) once the fix is in production?

@davidebbo
Copy link
Contributor

The fix is now fully deployed.

@felixpp
Copy link

felixpp commented Oct 22, 2019

I know this is outdated, but still, for people who may experience that issue, I just had the exact same problem for over 24h now, and I finally figured out by browsing KUDU that we had sketchy environment variables :

APPSETTING_WEBSITE_DISABLE_MSI=true
WEBSITE_DISABLE_MSI=true

Now, I have no idea where those environment variables came from in the first place, but removing these from our environment resolved the issue.

@ghost ghost locked as resolved and limited conversation to collaborators Jan 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants