InSpec verifier in Test Kitchen does not support the new reporter output

[burtlo]

Description

I started working on this learn chef rally module with pre-release of InSpec 2.0.x. In this module you use test kitchen with InSpec and kitchen-dokken.

At one point the module you demonstrate how you can set the format to JSON. I did that and execute:

-----> Starting Kitchen (v1.19.2)
-----> Verifying <default-ubuntu-1604>...
[DEPRECATED] The option --format is being is being deprecated and will be removed in inspec 3.0. Please use --reporter
[DEPRECATED] The option 'output' is being is being deprecated and will be removed in inspec 3.0. Please use --reporter name:path
[2018-02-19T00:38:50-06:00] WARN: URL target https://github.com/dev-sec/linux-baseline transformed to https://github.com/dev-sec/linux-baseline/archive/master.tar.gz. Consider using the git fetcher
       Loaded linux-baseline
       Finished verifying <default-ubuntu-1604> (0m2.64s).
-----> Kitchen is finished. (0m4.33s)

I see the deprecations so I switch the verifier from format to reporter.

      inspec_tests:
        - name: dev-sec/linux-baseline
      controls:
        - package-08
      format: json
      output: ./linux-baseline-results.json

I run verify again and see no file is generated.

I re-read the deprecation and and update the verifier to the following:

      inspec_tests:
        - name: dev-sec/linux-baseline
      controls:
        - package-08
      reporter: json:./linux-baseline-results.json

I run verify again and see no file is generated.

InSpec and Platform Version

$ inspec version
2.0.11

Commit: 8416c07a

[chris-rock]

This work is scheduled for this week @jquick

[jquick]

The old format should work fine but we should still update it via kitchen-inspec so we don't have deprecation.

verifier:
  name: inspec
  format: json
  output: json.out

[aaronlippold]

When the inspec_tests has two profiles - the old format is producing two json files but with no data. I will follow up with more info but give it a try on your side, with was with 1.51.x and 2.0.11 on my side.

[aaronlippold]

It seems to dump the json output to the screen as well.

[aaronlippold]

If possible when I load more than one profile, if I only have one output file for my reporter I would like to condense the output to that one file.

[jquick]

The new stable 2.0.17 should have all the fixes for this in it. The two profiles getting exported separately is news to me though. Ill have to take a look.

[aaronlippold]

Ok. Thanks. Let me know

[jquick]

I have confirmed this is working as expected in 1.51.21 and 2.0.17

[tolland]

Seems to be ignoring the output option, selecting the right formatter, but forcing stdout. This is quick work-around that gets junit and json working;
tolland/inspec@12208de

[jquick]

Thanks @tolland. Are you on version >= 1.51.21? A fix like yours has been added to more recent versions:
https://github.com/chef/inspec/blob/master/lib/inspec/base_cli.rb#L103

[tolland]

Ah I missed that.. yeah that fix is good!

[aaronlippold]

Should this issue be closed?

[jquick]

Yes, closing.

[kabakakao]

For me it is not working as expected:

    verifier:
      name: inspec
      reporter: json:./linux-baseline-results.json

This will be ignored and I got the cli output.

bundle exec inspec version
2.2.64

[stsully]

For me it is not working as expected:

    verifier:
      name: inspec
      reporter: json:./linux-baseline-results.json

I think kitchen is expecting it to be an array, so try to configure it like this:

    verifier:
      name: inspec
      reporter: 
        - json:./linux-baseline-results.json