Add nix language parser

[bsima]

This is a work in progress. Okay I think this is in good shape now, seems to work well in my usage and passes unit tests.

[masatake]

To docs/news/HEAD.rst, please add:

* Nix *optlib* by YOURNAME 

[masatake]

Thank you.

[bsima]

I updated the commit with your comments. Next I'll work on expanding the test case to actually test real-world nix code: I'll make a small library of functions and a couple builders with different formatting.

[masatake]

You may have to put "NixParser" here:

ctags/main/parsers_p.h

Line 135 in 8976ec3

MyrddinParser, \

.

After doing "make", you can run the test cases with "make units LANGUAGES=Nix".

[bsima]

I added some better test nix code, I tried to look at some of my personal nix code and copy a few formatting patterns that I use, which is similar to the style you see in upstream nixpkgs afaik. So I would expect this optlib to work reasonably well everywhere.

I thought I would need a multiline regex for the multilineFunc and multiline_attrset but I don't? It just worked? Maybe I don't understand regex as well as I thought?

Anyway sorry for taking so long to finish this. But I've been using this optlib in my personal code, which runs over nixpkgs and indeed finds most definitions I need, and my work code, which has thousands of lines of custom nix and it seems to work more often than not.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 21 lines in your changes missing coverage. Please review.

Project coverage is 85.39%. Comparing base (08e07dc) to head (8dfde48).
Report is 37 commits behind head on master.

Files	Patch %	Lines
optlib/nix.c	0.00%	21 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4020      +/-   ##
==========================================
- Coverage   85.43%   85.39%   -0.04%     
==========================================
  Files         235      236       +1     
  Lines       56727    56750      +23     
==========================================
  Hits        48462    48462              
- Misses       8265     8288      +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[masatake]

Could you rebase your chnages on the latest code?

[masatake]

As far as reading the test case, package is not tested at all.

[bsima]

doh, good catch, i fixed the regex here

[masatake]

The reason for rejecting short names looks weak.

One of the mottos of Universal Ctags design is to "prepare raw information to ctags client tools like vim when ctags doesn't know what's incorrect."

Rejecting can be done in a client tool reading the tags file.

[bsima]

ok, removed that part

[masatake]

The multiline regex meta parser is a toy in many situations.
The multi-table byte-oriented meta parser (--_tabledef-, and --_mtable-regex-) is the way to go.

I don't know the Nix language at all. However, I guess you may want to fill out the scope fields for tag entries. Without it, a client tool like Vim cannot draw a tree, which is useful for navigation.

struct point {
   int x;
   int y;
};

For the input of C language, ctags can emit the following tags:

point	input.c	/^struct point {$/;"	s	file:
x	input.c	/^  int x;$/;"	m	struct:point	typeref:typename:int	file:
y	input.c	/^  int y;$/;"	m	struct:point	typeref:typename:int	file:

From the tags output, x is a part of point.

  attrset = {
    foo = "bar";
  };

The current nix parser doesn't extract foo because the name is too short.
I guess an ideal Nix parser may emit:

attrest	input.nix	/^  attrset = {$/;"	a
foo	input.nix		;"	a	attr:attrest

Too many tags may not be a problem if the scope fields are filled out well.

However, these improvements can be made after merging this pull request.

[bsima]

Ah, scoped tags is a very good idea. It's something I would need if I want to
continue using tags in nixpkgs. Currently I just scan through the list of
duplicated tags for the correct one, but scopes is a much better solution. I
will read about scopes tomorrow.