Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic when creating Config object with Azure provider #238

Closed
amargherio opened this issue May 12, 2020 · 2 comments · Fixed by #394
Closed

Panic when creating Config object with Azure provider #238

amargherio opened this issue May 12, 2020 · 2 comments · Fixed by #394
Labels
bug Something isn't working config Kube config related

Comments

@amargherio
Copy link

Whenever I attempt to create a Config object (either via Config::infer() or Config::from_custom_kubeconfig(), I end up with the following output (full backtrace included).

It looks like the call to load_gcp is the last notable one before the unwind starts. The config I'm attempting to load (all of them, actually), are running Kube 1.16.7 in Azure AKS - the clusters are linked to AD.

I will work on getting better reproduction steps, but for now it seems to be loading a Kube config with a Azure data in the parsed AuthInfo.

Output and backtrace

thread 'main' panicked at 'no entry found for key', /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/collections/hash/map.rs:1023:9
stack backtrace:
   0:        0x101f541ff - backtrace::backtrace::libunwind::trace::hfff3154c85fa162b
                               at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/libunwind.rs:86
   1:        0x101f541ff - backtrace::backtrace::trace_unsynchronized::h4a9920525cd627b4
                               at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/mod.rs:66
   2:        0x101f541ff - std::sys_common::backtrace::_print_fmt::h972e000d0e4ed1fa
                               at src/libstd/sys_common/backtrace.rs:78
   3:        0x101f541ff - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hdfed66b3e2e8ce60
                               at src/libstd/sys_common/backtrace.rs:59
   4:        0x101f7886e - core::fmt::write::h9de7b13cfaa80b03
                               at src/libcore/fmt/mod.rs:1063
   5:        0x101f4d3a7 - std::io::Write::write_fmt::h9197012b9d65a9a3
                               at src/libstd/io/mod.rs:1426
   6:        0x101f5686a - std::sys_common::backtrace::_print::hb8e24d3c51344086
                               at src/libstd/sys_common/backtrace.rs:62
   7:        0x101f5686a - std::sys_common::backtrace::print::h93407cb3f8a6ecb3
                               at src/libstd/sys_common/backtrace.rs:49
   8:        0x101f5686a - std::panicking::default_hook::{{closure}}::h7b209cdc44c5777f
                               at src/libstd/panicking.rs:204
   9:        0x101f565ac - std::panicking::default_hook::hf135bc502e6f77db
                               at src/libstd/panicking.rs:224
  10:        0x101f56ed8 - std::panicking::rust_panic_with_hook::h7967810bc33e523b
                               at src/libstd/panicking.rs:470
  11:        0x101f56aa2 - rust_begin_unwind
                               at src/libstd/panicking.rs:378
  12:        0x101f8a21f - std::panicking::begin_panic::hc31d5034796a8de1
  13:        0x101f8a0aa - std::panicking::begin_panic::hc31d5034796a8de1
  14:        0x1016e7b94 - core::option::Option<T>::expect::h2dec13cd6a0959f9
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libcore/option.rs:347
  15:        0x101617f7c - <std::collections::hash::map::HashMap<K,V,S> as core::ops::index::Index<&Q>>::index::he319f16f8ecbc62e
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/collections/hash/map.rs:1023
  16:        0x101645514 - kube::config::file_config::AuthInfo::load_gcp::{{closure}}::h6a39c9023e63f71f
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/kube-0.34.0/src/config/file_config.rs:171
  17:        0x101655ff1 - <std::future::GenFuture<T> as core::future::future::Future>::poll::h7afab910415553ca
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:44
  18:        0x10165367b - std::future::poll_with_tls_context::h3cf622f34d6efc9b
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:102
  19:        0x10163a44f - kube::config::file_loader::ConfigLoader::load::{{closure}}::h999da559150140e6
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/kube-0.34.0/src/config/file_loader.rs:104
  20:        0x101654b61 - <std::future::GenFuture<T> as core::future::future::Future>::poll::h06ee47042af799ab
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:44
  21:        0x10165320b - std::future::poll_with_tls_context::h0f4dba94eead502f
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:102
  22:        0x101639164 - kube::config::file_loader::ConfigLoader::new_from_options::{{closure}}::h6f7df181959ef436
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/kube-0.34.0/src/config/file_loader.rs:51
  23:        0x101655351 - <std::future::GenFuture<T> as core::future::future::Future>::poll::h1bf733da1392b5b7
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:44
  24:        0x101653c6b - std::future::poll_with_tls_context::h8f57f74dc6bfa392
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:102
  25:        0x101622786 - kube::config::Config::from_kubeconfig::{{closure}}::h77aeea44c657d421
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/kube-0.34.0/src/config/mod.rs:177
  26:        0x101655081 - <std::future::GenFuture<T> as core::future::future::Future>::poll::h1625b642cb39dbb6
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:44
  27:        0x10165344b - std::future::poll_with_tls_context::h286839a4620654fe
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:102
  28:        0x10162374f - kube::config::Config::infer::{{closure}}::hc4a5ec8a7ac3f75e
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/kube-0.34.0/src/config/mod.rs:123
  29:        0x101655a61 - <std::future::GenFuture<T> as core::future::future::Future>::poll::h51760f0eb36bf5db
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:44
  30:        0x10165314b - std::future::poll_with_tls_context::h06329eedd74437c5
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:102
  31:        0x101633ca3 - kube::client::Client::try_default::{{closure}}::h070a16570e5533bb
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/kube-0.34.0/src/client/mod.rs:65
  32:        0x1016558f1 - <std::future::GenFuture<T> as core::future::future::Future>::poll::h4c3330c338bf812f
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:44
  33:        0x10165426b - std::future::poll_with_tls_context::hcfdd48f954cbb2e4
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:102
  34:        0x10161af9f - hpa_fixer::get_kube_client::{{closure}}::h64e6bbca89aec95e
                               at src/main.rs:125
  35:        0x101656671 - <std::future::GenFuture<T> as core::future::future::Future>::poll::hd6257d77086a615a
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:44
  36:        0x10165432b - std::future::poll_with_tls_context::hd60512efbdd750f0
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:102
  37:        0x1016183b9 - hpa_fixer::hpa_scale::{{closure}}::h642f3a9e17e3a71b
                               at src/main.rs:53
  38:        0x101654f87 - <std::future::GenFuture<T> as core::future::future::Future>::poll::h1220d68939d9c145
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:44
  39:        0x10165395c - std::future::poll_with_tls_context::h6447938b6aec294d
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:102
  40:        0x10161be05 - hpa_fixer::main::{{closure}}::h1028363ffeaab443
                               at src/main.rs:39
  41:        0x101656407 - <std::future::GenFuture<T> as core::future::future::Future>::poll::h94c090a2d549f963
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/future.rs:44
  42:        0x10162140d - tokio::runtime::enter::Enter::block_on::{{closure}}::ha4dedcec20b1dbc5
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.20/src/runtime/enter.rs:163
  43:        0x1016263a2 - tokio::coop::budget::{{closure}}::h12181b448a9ffd4d
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.20/src/coop.rs:97
  44:        0x1016419b9 - std::thread::local::LocalKey<T>::try_with::h2f64d990be737819
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/thread/local.rs:262
  45:        0x101641069 - std::thread::local::LocalKey<T>::with::he7bc7b724d2b5873
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/thread/local.rs:239
  46:        0x1016212a8 - tokio::coop::budget::h39cf039a7bbff41d
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.20/src/coop.rs:79
  47:        0x1016212a8 - tokio::runtime::enter::Enter::block_on::h3c62de9f6a4b65eb
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.20/src/runtime/enter.rs:163
  48:        0x10161d6fb - tokio::runtime::thread_pool::ThreadPool::block_on::hb8f79ff1a836c64f
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.20/src/runtime/thread_pool/mod.rs:82
  49:        0x101657f6e - tokio::runtime::Runtime::block_on::{{closure}}::hb98f0459e6e605f2
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.20/src/runtime/mod.rs:446
  50:        0x101623ece - tokio::runtime::context::enter::ha13c2c6a000c5e77
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.20/src/runtime/context.rs:72
  51:        0x101617721 - tokio::runtime::handle::Handle::enter::h14e6ee4eda90355a
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.20/src/runtime/handle.rs:71
  52:        0x101657eb3 - tokio::runtime::Runtime::block_on::hd6109c20ae83987a
                               at /Users/adam.margherio/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.20/src/runtime/mod.rs:441
  53:        0x101659095 - hpa_fixer::main::hfbc284000fd5a8cf
                               at src/main.rs:19
  54:        0x10165b78e - std::rt::lang_start::{{closure}}::h522d8e11305b330d
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/rt.rs:67
  55:        0x101f569a8 - std::rt::lang_start_internal::{{closure}}::hab79accac7befc7d
                               at src/libstd/rt.rs:52
  56:        0x101f569a8 - std::panicking::try::do_call::he58e98df0f664580
                               at src/libstd/panicking.rs:303
  57:        0x101f5e87b - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:86
  58:        0x101f573aa - std::panicking::try::h1c15a2595ce4d309
                               at src/libstd/panicking.rs:281
  59:        0x101f573aa - std::panic::catch_unwind::h06858616dd2b84f7
                               at src/libstd/panic.rs:394
  60:        0x101f573aa - std::rt::lang_start_internal::hfa21381d8ff23689
                               at src/libstd/rt.rs:51
  61:        0x10165b771 - std::rt::lang_start::h86e4f864d7727c21
                               at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/rt.rs:67
  62:        0x101659102 - hpa_fixer::main::hfbc284000fd5a8cf

Cargo.toml

[dependencies]
kube = "0.34.0"
kube-derive = "0.34.0"
k8s-openapi = { version = "0.8.0", default-features = false, features = ["v1_16"] }
base64 = "0.12.0"
chrono = { version = "0.4.11", features = ["serde"] }
tokio = { version = "0.2.18", features = ["full"] }
clap = "2.33.0"
log = "0.4.8"
env_logger = "0.7.1"
serde = { version = "1.0.106", features = ["derive"] }
serde_json = "1.0.51"
anyhow = "1.0"
@clux
Copy link
Member

clux commented May 14, 2020

So this is a failure in https://github.com/clux/kube-rs/blob/master/kube/src/config/file_config.rs#L171 , presumably it's because you have an access token without expiry. Is that correct? I thought the expiry was mandatory. But haven't checked with gcp myself. Haven't got time to dig into this right now, looks like maybe #84 has related info?

@clux clux added bug Something isn't working config Kube config related labels May 14, 2020
@amargherio
Copy link
Author

The auth block and access token do have an expiry on them - I've included a sanitized snippet below of the auth info from the config. I think what's holding everything up might be: https://github.com/clux/kube-rs/blob/2e2f51a5ee2b2201a3d957febf9cc20ece5d27fd/kube/src/config/file_config.rs#L174

I'm getting a completely filled out AuthInfo struct and it's correctly parsing everything from the YAML, but I think it's rolling over since it's assuming a GCP access token, and this one has been issued via Azure Active Directory for the AKS cluster in question.

users:
- name: *appropriate-user-name-for-cluster-user*
  user:
    auth-provider:
      config:
        access-token: *access-token-snipped, was present and valid as of post time*
        apiserver-id: *apiserver-id-snipped*
        client-id: *client-id-value-snipped*
        config-mode: "1"
        environment: AzurePublicCloud
        expires-in: "3599"
        expires-on: "1587508258"
        refresh-token: *refresh-token-snipped, value present*
        tenant-id: *azure-tenant-id-was-here*
      name: azure

I'm trying to dig around a bit and see what exactly kubectl is doing when it would make these auth calls - if I can get it narrowed down quickly, I'm up for PR'ing the Azure flow back in if it's beneficial.

@clux clux changed the title Panic when creating Config object Panic when creating Config object with Azure provider Feb 5, 2021
@clux clux linked a pull request Feb 5, 2021 that will close this issue
@kazk kazk linked a pull request Feb 7, 2021 that will close this issue
@clux clux closed this as completed in #394 Feb 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working config Kube config related
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants