[Bug] Azure Identity 1.13 Fails while calling MSAL

[marissa-df]

Library version used

4.65.0 and 4.66.0

.NET version

8.0

Scenario

Other - please specify

Is this a new or an existing app?

The app is in production, I haven't upgraded MSAL, but started seeing this issue

Issue description and reproduction steps

Upgrading to Azure.Identity 1.13 triggers token validation to start to fail. I opened a ticket over there and they told me that starting with v1.13 they have delegated token acquisition to MSAL. When getting an access token to pull keys from KeyVault the following exception gets throw:

---> Microsoft.Extensions.Configuration.AzureAppConfiguration.KeyVaultReferenceException: ManagedIdentityCredential authentication failed: The input string '2024-10-18T19:51:37.0000000+00:00' was not in a correct format.
See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot ErrorCode:'' ' Label:'Development' Etag:'KglaaAgZxV23Y-9AK8mdvqv1MDsaQ4pCjszEG83ZJjc' SecretIdentifier:'https://publicservicesdev.vault.azure.net/secrets/Audience'
---> Azure.Identity.AuthenticationFailedException: ManagedIdentityCredential authentication failed: The input string '2024-10-18T19:51:37.0000000+00:00' was not in a correct format.
See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot
---> System.FormatException: The input string '2024-10-18T19:51:37.0000000+00:00' was not in a correct format.
at System.Number.ThrowFormatException[TChar](ReadOnlySpan1 value) at System.Int64.Parse(String s, IFormatProvider provider) at Microsoft.Identity.Client.Utils.DateTimeHelpers.GetDurationFromNowInSeconds(String unixTimestampInFuture) at Microsoft.Identity.Client.OAuth2.MsalTokenResponse.ValidateManagedIdentityResult(ManagedIdentityResponse response) at Microsoft.Identity.Client.OAuth2.MsalTokenResponse.CreateFromManagedIdentityResponse(ManagedIdentityResponse managedIdentityResponse) at Microsoft.Identity.Client.Internal.Requests.ManagedIdentityAuthRequest.SendTokenRequestForManagedIdentityAsync(ILoggerAdapter logger, CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.ManagedIdentityAuthRequest.GetAccessTokenAsync(CancellationToken cancellationToken, ILoggerAdapter logger) at Microsoft.Identity.Client.Internal.Requests.ManagedIdentityAuthRequest.ExecuteAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.RequestBase.<>c__DisplayClass11_1.<<RunAsync>b__1>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Identity.Client.Utils.StopwatchService.MeasureCodeBlockAsync(Func1 codeBlock)
at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)
at Microsoft.Identity.Client.ApiConfig.Executors.ManagedIdentityExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenForManagedIdentityParameters managedIdentityParameters, CancellationToken cancellationToken)
at Azure.Identity.MsalManagedIdentityClient.AcquireTokenForManagedIdentityAsyncCore(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.MsalManagedIdentityClient.AcquireTokenForManagedIdentityAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.ManagedIdentityClient.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.ManagedIdentityCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)
at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.DefaultAzureCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.SetResultOnTcsFromCredentialAsync(TokenRequestContext context, TaskCompletionSource1 targetTcs, Boolean async, CancellationToken cancellationToken) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetAuthHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.TokenRequestState.GetCurrentHeaderValue(Boolean async, Boolean checkForCompletion, CancellationToken cancellationToken) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetAuthHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AuthenticateAndAuthorizeRequestAsync(HttpMessage message, TokenRequestContext context) at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallengeAsyncInternal(HttpMessage message, Boolean async) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async)
at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async)
at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.HttpPipeline.SendRequestAsync(Request request, CancellationToken cancellationToken) at Azure.Security.KeyVault.KeyVaultPipeline.SendRequestAsync(Request request, CancellationToken cancellationToken) at Azure.Security.KeyVault.KeyVaultPipeline.SendRequestAsync[TResult](RequestMethod method, Func1 resultFactory, CancellationToken cancellationToken, String[] path)
at Azure.Security.KeyVault.Secrets.SecretClient.GetSecretAsync(String name, String version, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureKeyVault.AzureKeyVaultSecretProvider.GetSecretValue(KeyVaultSecretIdentifier secretIdentifier, String key, String label, Logger logger, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureKeyVault.AzureKeyVaultKeyValueAdapter.ProcessKeyValue(ConfigurationSetting setting, Uri endpoint, Logger logger, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureKeyVault.AzureKeyVaultKeyValueAdapter.ProcessKeyValue(ConfigurationSetting setting, Uri endpoint, Logger logger, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.ProcessAdapters(ConfigurationSetting setting, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.PrepareData(Dictionary2 data, CancellationToken cancellationToken) at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.InitializeAsync(IEnumerable1 clients, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.TryInitializeAsync(IEnumerable1 clients, List1 startupExceptions, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.LoadAsync(Boolean ignoreFailures, CancellationToken cancellationToken)
at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.Load()
at Microsoft.Extensions.Configuration.ConfigurationManager.AddSource(IConfigurationSource source)
at Microsoft.Extensions.Configuration.ConfigurationManager.Microsoft.Extensions.Configuration.IConfigurationBuilder.Add(IConfigurationSource source)
at Microsoft.Extensions.Configuration.AzureAppConfigurationExtensions.AddAzureAppConfiguration(IConfigurationBuilder configurationBuilder, Action`1 action, Boolean optional)
--- End of inner exception stack trace ---

This starts to happen after upgrading Azure.Identity with no other changes to the code base. Azure.Identity deps 4.65.0 and I have tried pinning version 4.66.0 both versions exhibit the same behavior. This project uses Docker and is based on the mcr.microsoft.com/dotnet/aspnet:8.0 image. If I start this project outside of Docker the access token is retrieved as expected. I suspect that it's not detecting the OS correctly when run inside the container and is trying to parse a Unix time rather than the RFC3336 formatted datetime.

Relevant code snippets

public static Action<AzureAppConfigurationOptions> GetMyStandardAzureAppConfigurationOptions(
      string connectionString,
      string environmentLabel,
      IEnumerable<string> applicationLabels,
      string sentinel)
    {
      return (Action<AzureAppConfigurationOptions>) (options =>
      {
        options.Connect(connectionString).ConfigureKeyVault((Action<AzureAppConfigurationKeyVaultOptions>) (kv => kv.SetCredential((TokenCredential) new DefaultAzureCredential()))).ConfigureRefresh((Action<AzureAppConfigurationRefreshOptions>) (refreshOptions => refreshOptions.Register(sentinel, true)));
        options.Select("B2C:*", environmentLabel);
        options.Select("Entra:*", environmentLabel);
        foreach (string applicationLabel in applicationLabels)
          options.Select(applicationLabel + ":*", environmentLabel);
      });
    }

Expected behavior

The token should be retrieved successfully and the KeyVault secrets should be retrieved.

Identity provider

Microsoft Entra ID (Work and School accounts and Personal Microsoft accounts)

Regression

No response

Solution and workarounds

No response

[bgavrilMS]

@marissa-df - thanks for reporting this. You'll need to downgrade Azure.Identity to mitigate it.

CC @christothes

[rayluo]

Hi, @marissa-df , you mentioned docker. Were you running your containerized web app with Azure App Service? I am trying to better understand where this issue happened.

[gladjohn]

@marissa-df Azure SDK version https://www.nuget.org/packages/Azure.Identity/1.13.1 has been released