running inspec check on a profile that references the latest version of the resource pack fails

[srb3]

Previously when running inspec check (or inspec archive) on a profile that depended on the inspec azure resource pack it would work without any errors. When running check or archive on a profile that depends on the latest version of the resource pack it fails because the azure backed requires environment variables to be set.

steveb@chef-x360:[cloud_scans]$ grep url test-profile-azure-level-2/inspec.yml
    url: https://github.com/inspec/inspec-azure/archive/v1.16.0.tar.gz
steveb@chef-x360:[cloud_scans]$ grep url test-profile-azure-level-1/inspec.yml
    url: https://github.com/inspec/inspec-azure/archive/v1.32.1.tar.gz


steveb@chef-x360:[cloud_scans]$ inspec check test-profile-azure-level-2/
W, [2020-09-30T20:21:35.483584 #400465]  WARN -- : Control sec-azure-control-1 has no tests defined
W, [2020-09-30T20:21:35.483626 #400465]  WARN -- : Control sec-azure-control-2 has no descriptions
W, [2020-09-30T20:21:35.483638 #400465]  WARN -- : Control sec-azure-control-2 has no tests defined
W, [2020-09-30T20:21:35.483647 #400465]  WARN -- : Control sec-azure-control-3 has no descriptions
W, [2020-09-30T20:21:35.483665 #400465]  WARN -- : Control sec-azure-control-3 has no tests defined
W, [2020-09-30T20:21:35.483674 #400465]  WARN -- : Control sec-azure-control-4 has no descriptions
W, [2020-09-30T20:21:35.483683 #400465]  WARN -- : Control sec-azure-control-4 has no tests defined
Location :   test-profile-azure-level-2/
Profile :    test-profile-azure-level2
Controls :   4
Timestamp :  2020-09-30T20:21:35+01:00
Valid :      true

 !  test-profile-azure-level-2/controls/sec_1_50_01.rb:6: Control sec-azure-control-1 has no tests defined
 !  test-profile-azure-level-2/controls/sec_1_50_01.rb:31: Control sec-azure-control-2 has no descriptions
 !  test-profile-azure-level-2/controls/sec_1_50_01.rb:31: Control sec-azure-control-2 has no tests defined
 !  test-profile-azure-level-2/controls/sec_1_50_01.rb:47: Control sec-azure-control-3 has no descriptions
 !  test-profile-azure-level-2/controls/sec_1_50_01.rb:47: Control sec-azure-control-3 has no tests defined
 !  test-profile-azure-level-2/controls/sec_1_50_01.rb:65: Control sec-azure-control-4 has no descriptions
 !  test-profile-azure-level-2/controls/sec_1_50_01.rb:65: Control sec-azure-control-4 has no tests defined

Summary:     0 errors, 7 warnings

steveb@chef-x360:[cloud_scans]$ inspec check test-profile-azure-level-1/
[2020-09-30T20:21:41+01:00] WARN: DEPRECATION: `azurerm_resource_groups` uses the new resource `azure_resource_groups` under the hood. azurerm_resource_groups will be deprecated soon and it is advised to switch to the fully backward compatible new resource. Please see the documentation for the additional features available.
Traceback (most recent call last):
        31: from /opt/chef-workstation/bin/inspec:363:in `<main>'
        30: from /opt/chef-workstation/bin/inspec:363:in `load'
        29: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-bin-4.19.0/bin/inspec:11:in `<top (required)>'
        28: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/base_cli.rb:35:in `start'
        27: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
        26: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
        25: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
        24: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
        23: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/cli.rb:108:in `check'
        22: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:425:in `check'
        21: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:455:in `controls_count'
        20: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:196:in `params'
        19: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:612:in `load_params'
        18: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:619:in `load_checks_params'
        17: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:205:in `collect_tests'
        16: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:205:in `each'
        15: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:209:in `block in collect_tests'
        14: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:154:in `load_control_file'
        13: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:170:in `load_with_context'
        12: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:170:in `instance_eval'
        11: from test-profile-azure-level-1/controls/sec_1_50_01.rb:2:in `load_with_context'
        10: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:252:in `block (2 levels) in add_registry_methods'
         9: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:252:in `new'
         8: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/resource.rb:118:in `initialize'
         7: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/resource.rb:54:in `supersuper_initialize'
         6: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/resource.rb:119:in `block in initialize'
         5: from libraries/azure_resource_groups.rb:77:in `initialize'
         4: from libraries/azure_resource_groups.rb:24:in `initialize'
         3: from libraries/azure_generic_resources.rb:16:in `initialize'
         2: from libraries/azure_backend.rb:48:in `initialize'
         1: from libraries/azure_backend.rb:48:in `new'
libraries/backend/azure_connection.rb:62:in `initialize': The following must be set in the Environment: [:tenant_id, :client_id, :client_secret, :subscription_id].Provided: {:tenant_id=>nil, :client_id=>nil, :client_secret=>nil, :subscription_id=>nil}
        30: from /opt/chef-workstation/bin/inspec:363:in `<main>'
        29: from /opt/chef-workstation/bin/inspec:363:in `load'
        28: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-bin-4.19.0/bin/inspec:11:in `<top (required)>'
        27: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/base_cli.rb:35:in `start'
        26: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
        25: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
        24: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
        23: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
        22: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/cli.rb:108:in `check'
        21: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:425:in `check'
        20: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:455:in `controls_count'
        19: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:196:in `params'
        18: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:612:in `load_params'
        17: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:619:in `load_checks_params'
        16: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:205:in `collect_tests'
        15: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:205:in `each'
        14: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile.rb:209:in `block in collect_tests'
        13: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:154:in `load_control_file'
        12: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:170:in `load_with_context'
        11: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:170:in `instance_eval'
        10: from test-profile-azure-level-1/controls/sec_1_50_01.rb:2:in `load_with_context'
         9: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:252:in `block (2 levels) in add_registry_methods'
         8: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/profile_context.rb:252:in `new'
         7: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/resource.rb:118:in `initialize'
         6: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/resource.rb:54:in `supersuper_initialize'
         5: from /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.19.0/lib/inspec/resource.rb:119:in `block in initialize'
         4: from libraries/azure_resource_groups.rb:77:in `initialize'
         3: from libraries/azure_resource_groups.rb:24:in `initialize'
         2: from libraries/azure_generic_resources.rb:16:in `initialize'
         1: from libraries/azure_backend.rb:47:in `initialize'
libraries/azure_backend.rb:52:in `rescue in initialize': HTTP Client Error.The following must be set in the Environment: [:tenant_id, :client_id, :client_secret, :subscription_id].
Provided: {:tenant_id=>nil, :client_id=>nil, :client_secret=>nil, :subscription_id=>nil}

[omerdemirok]

@srb3, can you check if the following are set as environment variables?

AZURE_SUBSCRIPTION_ID=
AZURE_CLIENT_ID=
AZURE_TENANT_ID=
AZURE_CLIENT_SECRET=

Seems like they are not set:

The following must be set in the Environment: [:tenant_id, :client_id, :client_secret, :subscription_id].
Provided: {:tenant_id=>nil, :client_id=>nil, :client_secret=>nil, :subscription_id=>nil}

[gsreynolds]

Hi @omerdemirok, this has been identified as an issue with the profile causing inspec check to blow up without the environment variables set.

An InSpec input existed that was loading the resource group names as the default value:

resource_groups = input('provided_resource_group', value: azure_resource_groups.names)

Removing this line and just looping over all resource group names fixed it up.

-  resource_groups.each do |resource_group|
+  azure_resource_groups.names.each do |resource_group|

cc: @srb3

[omerdemirok]

Thanks for that @gsreynolds and @srb3.
I added the exact same line in azure_resourece_groups integration tests now to confirm the fix.
Hopefully, this will be resolved when the PR is merged. 🤞

[gsreynolds]

Cool thanks @omerdemirok! In the profile in question we've just removed the input since it's unnecessary there.

[omerdemirok]

@gsreynolds, checking the presence of credentials was causing issues in the unit test as well, and inspec check should pass for a valid use case. Now, we don't check credentials until authenticating with the API.
CC: @rmoles