Handling comments

[samrjenkins]

Yalphabetize does not currently preserve comments when reordering a YAML file.

Ruby's Psych parser does not parse comments into the AST.

I have been unable to figure out a reliable way of knowing where a comment should be reinserted into the YAML file after alphabetisation. For example:
ORIGINAL CODE

# A comment
Bananas: 2
Apples: 1

POSSIBLE OPTIONS AFTER ALPHABETISATION

# Sometimes the comment might belong at the top of the file
Apples: 1
Bananas: 2

Apples: 1
# Sometimes the comment is coupled to the line of code beneath it
Bananas: 2

[samrjenkins]

Let us address inline comments first
Must be able to handle comments of form

a_key: a_value # a comment

Must ignore # within strings:

a_key: '# not a comment'

[rotten]

I'd argue, after leaving inline comments with the line they are on - lets keep comment blocks with the line of code that immediately follows it. The developer should be able to move the block of comments that were supposed to be on the top of the file back to the top of the file. No one is going to run this without carefully looking over the result it produces.

The block at the top of the file is only one block of comments after all.

Of course you could say, keep all comment blocks with the line that immediately follows except the comment block at the top of the file, that stays where it is. And then the developer has to move it back down to where it was.

--

Some sort of diff or dry-run feature might also help to identify things that are being moved in ways you didn't expect or desire.

--

Stripping out the comments is not so great. One of the main reasons we like yaml over json is that we can add comments.

[samrjenkins]

@rotten there has been a development here. There is a new library which extends the Ruby YAML parser to preserve comments. Currently the library supports preservation of newline comments only but there is an open PR to also preserve line-end comments.

Please try my implementation in this branch/PR and see what you think.

[rotten]

Thanks! I'll give it a try and report back.

[rotten]

I pulled your branch, but when I try to run it:

rotten@rotten-am-laptop src/yalphabetize/bin $  git status
On branch handle-newline-comments
Your branch is up to date with 'origin/handle-newline-comments'.

nothing to commit, working tree clean

rotten@rotten-laptop src/yalphabetize/bin $  ./yalphabetize --help
/home/rotten/src/yalphabetize/lib/yalphabetize.rb:6:in `require_relative': /home/rotten/src/yalphabetize/lib/yalphabetize/file_yalphabetizer.rb:36: syntax error, unexpected ')' (SyntaxError)
...eam_node, order_checker_class:).offences?
...                              ^
/home/rotten/src/yalphabetize/lib/yalphabetize/file_yalphabetizer.rb:40: syntax error, unexpected ')'
...eam_node, order_checker_class:).call
...                              ^
	from /home/rotten/src/yalphabetize/lib/yalphabetize.rb:6:in `<top (required)>'
	from ./yalphabetize:4:in `require_relative'
	from ./yalphabetize:4:in `<main>'

I get the same error when I try it against a yml file (without the --help).

[rotten]

Maybe I'm on the wrong branch?

[samrjenkins]

@rotten please could you check your Ruby version? Be aware that Ruby 3.0 reached EOL on March 31st.

The latest version of yalphabetize now only supports Ruby 3.1-3.3.

It looks like you are running an older Ruby version which cannot interpret this modern Ruby syntax.

[rotten]

Indeed it appears so. I was running the default ruby version that comes with Ubuntu 22.04 as an apt package. At the moment it is ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux-gnu].

I switched to the snap version, which gets me ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux].

That worked. I tried it on a fairly large, disorganized file. The comments seemed to track with the relevant yaml lines. I didn't find any out of place except for the top most comment in the file and that was easy enough to move back up. Much better than before. :-)

The only thing I noticed is it pulled out all of the blank lines. I often use blank lines to visually separate sections in the yaml. Is a blank line really a different form of comment? I don't know. I can easily go through and put the blank lines back in after running yalphabetize by hand. I think that would be a blocker for me for using it as a linter since I'm kind of passionate about my blank lines. Blank lines are much easier to fix than copy and pasting comments back in from a prior commit. I really don't mind. It forces me to go through and inspect the changes that were applied, which is actually a good thing.

Running it by hand is pretty awesome and a huge win. I want to go through and fix all the yamls now. :-) Thanks for this!

[samrjenkins]

Thanks for the feedback. It’s great to hear that there are no obvious bugs. I’ve added it to the README but the trick to keep comments at the top of the yaml file is to place them above the start of the document:
https://github.com/samrjenkins/yalphabetize/pull/127/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R216
Otherwise the comment will remain above the line of yaml that it originally came before.

I’ve been thinking about how to preserve blank lines but it’s definitely beyond the scope of this issue.

It’s not immediately clear to me how the preservation of blank lines would behave: the blank line acts as a delimiter between two sections of the file. These sections may no longer be adjacent following the alphabetisation. Where does the blank line belong in this scenario?

I would love to know your thoughts. Perhaps you could open a separate issue for this.

[rotten]

I think a good starting place for blank lines is to treat them the same as your PR treats comments. It may not be perfect, but it should get a developer moving in the right direction for building a readable yaml file from a jumbled mess of one.

I'm happy to make another issue about that topic if you'd like.

[samrjenkins]

Thanks @rotten.

A new issue would be great.

If you are able to add examples, that would also be very helpful.

[samrjenkins]

Wrt comments, I am working on a solution to support line-end comments as well as new-line comments