Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parsing plans in unit tests fail without being caught #307

Closed
danielparks opened this issue Sep 25, 2022 · 5 comments
Closed

Parsing plans in unit tests fail without being caught #307

danielparks opened this issue Sep 25, 2022 · 5 comments
Labels

Comments

@danielparks
Copy link
Contributor

Describe the Bug

It currently fails to parse the plan in spec/unit/puppet-strings/json_spec.rb and fails with error [error]: Failed to parse (stdin): Syntax error at 'param1' (line: 5, column: 19).

Adding a new unit test to parse a plan also fails.

However, it is able to parse plans in the wild, e.g. https://github.com/puppetlabs/puppetlabs-pam_tools/blob/main/REFERENCE.md#plans-1 . I suppose it’s possible that it has stopped working since that was generated; I haven’t yet checked.

This is an offshoot of #302; it seemed like this might be a bigger issue than just warnings during tests so I figured I should split it out.

Expected Behavior

It should parse a plan successfully without generating failures and generate documentation for it.

Steps to Reproduce

Steps to reproduce the behavior:

  1. bundle exec rake spec

Environment

  • ruby 3.1.2p20
  • macOS 12.6.0
  • Current main branch (dba16ed)
@danielparks
Copy link
Contributor Author

I was sort of wrong about the second test. It was only failing because I was only running one file. Running all the tests made it work.

It turns out I can fix the original error by running this code at the top of the file (spec/unit/puppet-strings/json_spec.rb):

YARD::Parser::SourceParser.parse(File.expand_path("../../fixtures/plans/plan.pp", __dir__))
PuppetStrings::Markdown.generate

WTF.

@danielparks
Copy link
Contributor Author

Oh, or alternatively,

YARD::Parser::SourceParser.parse(File.expand_path("../../fixtures/plans/plan.pp", __dir__))
YARD::Registry.clear

To be clear, both lines are required to make it work, though they don’t have to be in that order.

@danielparks danielparks changed the title Does not parse plans in tests Parsing plans in unit tests fail without being caught Sep 25, 2022
@danielparks
Copy link
Contributor Author

Looks like this issue has something to do with the parser checking file paths to see if they match %r{^plans|/plans/}. If they do it might set Puppet[:tasks] = true.

if @file.to_s.match(/^plans|\/plans\//)
if Puppet::Util::Package.versioncmp(Puppet.version, "5.0.0") < 0
log.warn "Skipping #{@file}: Puppet Plans require Puppet 5 or greater."
return
end
Puppet[:tasks] = true if Puppet.settings.include?(:tasks)
end

If I add Puppet[:tasks] = true to the top of spec/unit/puppet-strings/json_spec.rb it fixes the issue.

danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 26, 2022
Previously, parsing of Puppet plan code failed in json_spec.rb because
it provide a file name to the parser, which prevented the parser from
enabling task support in Puppet, which prevented the parser from
recognizing a plan.

This turns on task support in Puppet before trying to parse a plan in
json_spec.rb.
danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 26, 2022
Previously, the tests for `PuppetStrings::Json` (in json_spec.rb) parsed
a variety of code samples and then queried YARD to test the results. If
a sample failed to parse, an error might be outputted (see [issue
puppetlabs#307][puppetlabs#307]), but no failure was registered by the testing harness.

This adds simple checks to verify that the parses actually succeed.

Unfortunately, one of the samples currently fails to parse, so this
commit introduces a testing error. The subbsequent commit will fix it.

[puppetlabs#307]: puppetlabs#307
danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 26, 2022
Previously, parsing of Puppet plan code failed in json_spec.rb because
it provide a file name to the parser, which prevented the parser from
enabling task support in Puppet, which prevented the parser from
recognizing a plan.

This turns on task support in Puppet before trying to parse a plan in
json_spec.rb.
@danielparks
Copy link
Contributor Author

I submitted PR #311 for this, but I keep vacillating on whether or not it’s the right approach. It would be less likely to surprise people if we just moved the Puppet[:tasks] = true if Puppet.settings.include?(:tasks) line above outside of the if statement. In other words, always enable task support if it’s available.

I’m not sure why you would want tasks support off; I have a hard time imaging that it would make much difference since it gets turned on automatically by this code in pretty much all practical use cases aside from testing.

danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 26, 2022
Previously, the parsing code activated task support in Puppet when it
detected that it was parsing a file in a plans/ directory. This casued
the JSON spec test for plans to fail because it parsed a string, which
naturally had no file name to check.

This turns on task support (and thus enables parsing plans) whenever the
parser starts (assuming that task support is available).

This helps avoid surprises to future users who need to parse plans from
ruby code, such as in tests.
danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 26, 2022
Previously, parsing of Puppet plan code failed in json_spec.rb because
it didn’t provide a file name to the parser, which prevented the parser
from enabling task support in Puppet, which prevented the parser from
recognizing a plan.

This turns on task support in Puppet before trying to parse a plan in
json_spec.rb.
danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 26, 2022
Previously, the tests for `PuppetStrings::Json` (in json_spec.rb) parsed
a variety of code samples and then queried YARD to test the results. If
a sample failed to parse, an error might be outputted (see [issue
puppetlabs#307][puppetlabs#307]), but no failure was registered by the testing harness.

This adds simple checks to verify that the parses actually succeed.

Unfortunately, one of the samples currently fails to parse, so this
commit introduces a testing error. The subbsequent commit will fix it.

[puppetlabs#307]: puppetlabs#307
danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 26, 2022
Previously, the parsing code activated task support in Puppet when it
detected that it was parsing a file in a plans/ directory. This casued
the JSON spec test for plans to fail because it parsed a string, which
naturally had no file name to check.

This turns on task support (and thus enables parsing plans) whenever the
parser starts (assuming that task support is available).

This helps avoid surprises to future users who need to parse plans from
ruby code, such as in tests.
danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 26, 2022
Previously, the tests for `PuppetStrings::Json` (in json_spec.rb) parsed
a variety of code samples and then queried YARD to test the results. If
a sample failed to parse, an error might be outputted (see [issue
puppetlabs#307][puppetlabs#307]), but no failure was registered by the testing harness.

This adds simple checks to verify that the parses actually succeed.

Unfortunately, one of the samples currently fails to parse, so this
commit introduces a testing error. The subbsequent commit will fix it.

[puppetlabs#307]: puppetlabs#307
danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 26, 2022
Previously, parsing of Puppet plan code failed in json_spec.rb because
it didn’t provide a file name to the parser, which prevented the parser
from enabling task support in Puppet, which prevented the parser from
recognizing a plan.

This turns on task support in Puppet before trying to parse a plan in
json_spec.rb.
danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 29, 2022
Previously, the tests for `PuppetStrings::Json` (in json_spec.rb) parsed
a variety of code samples and then queried YARD to test the results. If
a sample failed to parse, an error might be outputted (see [issue
puppetlabs#307][puppetlabs#307]), but no failure was registered by the testing harness.

This adds simple checks to verify that the parses actually succeed.

Unfortunately, one of the samples currently fails to parse, so this
commit introduces a testing error. The subbsequent commit will fix it.

[puppetlabs#307]: puppetlabs#307
danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 29, 2022
Previously, parsing of Puppet plan code failed in json_spec.rb because
it didn’t provide a file name to the parser, which prevented the parser
from enabling task support in Puppet, which prevented the parser from
recognizing a plan.

This turns on task support in Puppet before trying to parse a plan in
json_spec.rb.
danielparks added a commit to danielparks/puppet-strings that referenced this issue Sep 29, 2022
Previously, the parsing code activated task support in Puppet when it
detected that it was parsing a file in a plans/ directory. This casued
the JSON spec test for plans to fail because it parsed a string, which
naturally had no file name to check.

This turns on task support (and thus enables parsing plans) whenever the
parser starts (assuming that task support is available).

This helps avoid surprises to future users who need to parse plans from
ruby code, such as in tests.
chelnak added a commit that referenced this issue Sep 29, 2022
@danielparks
Copy link
Contributor Author

Fixed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant