403 Forbidden

[donwlewis]

I configured my Chef Server and Chef Compliance instances for integration. When I attempting to run an audit cookbook this is the error message I am getting:

================================================================================
    Error executing action `fetch` on resource 'compliance_profile[linux]'
    ================================================================================

    Net::HTTPServerException
    ------------------------
    403 "Forbidden"

    Cookbook Trace:
    ---------------
    /var/chef/cache/cookbooks/audit/libraries/server_api.rb:23:in `binmode_streaming_request'
    /var/chef/cache/cookbooks/audit/libraries/profile.rb:55:in `block (2 levels) in <class:ComplianceProfile>'
    /var/chef/cache/cookbooks/audit/libraries/profile.rb:46:in `block in <class:ComplianceProfile>'

    Resource Declaration:
    ---------------------
    # In /var/chef/cache/cookbooks/audit/recipes/default.rb

     25:   compliance_profile p do
     26:     owner o
     27:     action [:fetch, :execute]
     28:   end
     29: end

    Compiled Resource:
    ------------------
    # Declared in /var/chef/cache/cookbooks/audit/recipes/default.rb:25:in `block in from_file'

    compliance_profile("linux") do
      action [:fetch, :execute]
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      declared_type :compliance_profile
      cookbook_name "audit"
      recipe_name "default"
      owner "base"
      profile "linux"
    end


Running handlers:
[2016-04-08T16:13:15-07:00] ERROR: Running exception handlers
Running handlers complete
[2016-04-08T16:13:15-07:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 05 seconds
[2016-04-08T16:13:15-07:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2016-04-08T16:13:15-07:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2016-04-08T16:13:15-07:00] ERROR: compliance_profile[linux] (audit::default line 25) had an error: Net::HTTPServerException: 403 "Forbidden"
[2016-04-08T16:13:15-07:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Both servers are able to access each other over 443.

[srenatus]

Both servers are able to access each other over 443.

While this is a necessary, it's unfortunately not sufficient.

1. Can you look into your chef-server's chef_gate logs? (/var/log/opscode/chef_gate/current)? It's verifying the request's signature before chef-server's nginx forwards the request.
2. If you only see HTTP 200 in there, it might be that your chef-server's chef_gate and your chef-compliance instance don't properly share the same "shared secret". Please ensure that next, e.g. by checking the configuration, or by looking into chef-compliance's core's logs (/var/log/chef-compliance/core/current).

Thanks!

[JTabel]

I actually have the same issue, during converge the audit cookbook simply gives error 403 "Forbidden". My chef_gate log looks like this:

2016-05-09_14:47:09.15733 [GIN] 2016/05/09 - 16:47:09 | 400 | 8.515589ms | 10.4.9.117 | POST /compliance/organizations/$CHEF_SERVER_URL/inspec 2016-05-09_14:47:09.15735 Error #01: Couldn't find principal for $NODE_NAME (orgs $CHEF_SERVER_URL)

I also completely reinstalled compliance (still evaluating), but this did not change anything. Is there any log on the compliance server, that could help with this?

[srenatus]

@JTabel this is a stab in the dark, but does your node in question has a node_name != client_name? The described code path in chef_gate does this: given a signed request coming from a chef-client, it fetches the client's public key using GET /principals/NAME, and verifies the signature. So if to figure out what is wrong here, you could use chef-shell (or any chef API library you like using) with the pivotal key of chef_gate's chef-server, and lookup that endpoint for your node's name.

[donwlewis]

I ended up cleansing the install, upgrading to the latest version of compliance, and re-running the integration steps and was able to get it to work.

[atul86244]

I am facing the same issue while running the audit cookbook. I am using the latest version of compliance, chef server and inspec. Please help.

Logs below:

tail -f  /var/log/opscode/chef_gate/current
2016-05-16_13:21:09.99935 [GIN] 2016/05/16 - 13:21:09 | 200 |   13.777166ms | 192.168.1.143 |   POST    /compliance/organizations/demo/inspec
2016-05-16_13:24:11.95138 [GIN] 2016/05/16 - 13:24:11 | 200 |   27.596463ms | 192.168.1.117 |   GET     /orgs
2016-05-16_13:24:12.15772 [GIN] 2016/05/16 - 13:24:12 | 200 |   61.234133ms | 192.168.1.117 |   GET     /orgs
2016-05-16_13:24:12.27839 [GIN] 2016/05/16 - 13:24:12 | 200 |    23.54308ms | 192.168.1.117 |   GET     /orgs
2016-05-16_13:24:12.32930 [GIN] 2016/05/16 - 13:24:12 | 200 |   19.725617ms | 192.168.1.117 |   GET     /orgs
2016-05-16_13:24:12.38496 [GIN] 2016/05/16 - 13:24:12 | 200 |   20.564337ms | 192.168.1.117 |   GET     /orgs
2016-05-16_13:24:12.48848 [GIN] 2016/05/16 - 13:24:12 | 200 |   18.645486ms | 192.168.1.117 |   GET     /orgs
2016-05-16_13:24:12.56252 [GIN] 2016/05/16 - 13:24:12 | 200 |   22.251111ms | 192.168.1.117 |   GET     /orgs
2016-05-16_13:24:12.70535 [GIN] 2016/05/16 - 13:24:12 | 200 |   66.161231ms | 192.168.1.117 |   GET     /orgs
2016-05-16_13:24:12.87906 [GIN] 2016/05/16 - 13:24:12 | 200 |   20.658468ms | 192.168.1.117 |   GET     /orgs
2016-05-16_13:31:58.96585 [GIN] 2016/05/16 - 13:31:58 | 200 |   12.492756ms | 192.168.1.143 |   GET     /compliance/organizations/demo/owners/base/compliance/ssh/tar
2016-05-16_13:31:59.08582 [GIN] 2016/05/16 - 13:31:59 | 200 |   17.407865ms | 192.168.1.143 |   GET     /compliance/organizations/demo/owners/base/compliance/linux/tar
2016-05-16_13:31:59.23695 [GIN] 2016/05/16 - 13:31:59 | 200 |   16.617363ms | 192.168.1.143 |   POST    /compliance/organizations/demo/inspec

tail -f /var/log/chef-compliance/core/current
2016-05-16_13:24:12.59512 13:24:12.595 DEB => ID of user admin changed: fb3f7e81-dd53-43da-7f49-0efbd51c4942 -> 6c0dd10b-0ac8-4bc3-aa53-fd348a8734d9 (resetting)
2016-05-16_13:24:12.71378 13:24:12.713 DEB => orgs for user admin (fb3f7e81-dd53-43da-7f49-0efbd51c4942): []string{"demo"}
2016-05-16_13:24:12.72114 13:24:12.721 DEB => Authenticated user: &{PasswordHash: Login:admin Name:admin IsOrg:false Source:{String: Valid:false} UUID:{ID:fb3f7e81-dd53-43da-7f49-0efbd51c4942}}
2016-05-16_13:24:12.72137 [GIN] 2016/05/16 - 13:24:12 | 200 |  128.414153ms | 192.168.1.135 |   GET     /server/config
2016-05-16_13:24:12.80786 13:24:12.807 ERR => DB error: sql: no rows in result set
2016-05-16_13:24:12.80959 13:24:12.809 DEB => ID of user admin changed: fb3f7e81-dd53-43da-7f49-0efbd51c4942 -> 6c0dd10b-0ac8-4bc3-aa53-fd348a8734d9 (resetting)
2016-05-16_13:24:12.88667 13:24:12.886 DEB => orgs for user admin (fb3f7e81-dd53-43da-7f49-0efbd51c4942): []string{"demo"}
2016-05-16_13:24:12.88842 13:24:12.888 DEB => Authenticated user: &{PasswordHash: Login:admin Name:admin IsOrg:false Source:{String: Valid:false} UUID:{ID:fb3f7e81-dd53-43da-7f49-0efbd51c4942}}
2016-05-16_13:24:12.88880 13:24:12.888 ERR => DB error: sql: no rows in result set
2016-05-16_13:24:12.89010 [GIN] 2016/05/16 - 13:24:12 | 200 |   85.015616ms | 192.168.1.135 |   GET     /owners/adcf5c2c-cda8-483f-4262-84f7b0c1555a/keys
2016-05-16_13:31:59.01786 13:31:59.017 ERR => Authentication: %!(EXTRA *errors.errorString=missing Authorization header)
2016-05-16_13:31:59.01789 13:31:59.017 ERR => Parsing token from request:%!(EXTRA *jwt.ValidationError=No shared secret passed through the environment (CHEF_GATE_COMPLIANCE_SECRET))
2016-05-16_13:31:59.01790 [GIN] 2016/05/16 - 13:31:59 | 401 |     201.964µs | 192.168.1.124 |   GET     /chef/organizations/demo/owners/base/compliance/ssh/tar
2016-05-16_13:31:59.12812 13:31:59.127 ERR => Authentication: %!(EXTRA *errors.errorString=missing Authorization header)
2016-05-16_13:31:59.12815 13:31:59.127 ERR => Parsing token from request:%!(EXTRA *jwt.ValidationError=No shared secret passed through the environment (CHEF_GATE_COMPLIANCE_SECRET))
2016-05-16_13:31:59.12821 [GIN] 2016/05/16 - 13:31:59 | 401 |     295.419µs | 192.168.1.124 |   GET     /chef/organizations/demo/owners/base/compliance/linux/tar
2016-05-16_13:31:59.29408 13:31:59.293 ERR => Authentication: %!(EXTRA *errors.errorString=missing Authorization header)
2016-05-16_13:31:59.29410 13:31:59.293 ERR => Parsing token from request:%!(EXTRA *jwt.ValidationError=No shared secret passed through the environment (CHEF_GATE_COMPLIANCE_SECRET))
2016-05-16_13:31:59.29411 [GIN] 2016/05/16 - 13:31:59 | 401 |      371.77µs | 192.168.1.124 |   POST    /chef/organizations/demo/inspec

[atul86244]

I was able to fix this by restarting the compliance core service.

chef-compliance-ctl restart core

[chris-rock]

Thanks all for the feedback. Please reopen the issue if it persists.