EDoc: EEP-48 doc chunk generation #2803
Conversation
|
This is amazing work @erszcz! ❤️ One of the things to discuss, probably for after this PR is merged, is to deprecate the tags |
There was a problem hiding this comment.
I've done a cursory review and I think all in it it looks very good. Most of the comments are about backward-compatibility and documentation.
It would be nice if the changes of @spec to -spec would be a separate commit from everything else so that it is easier to see that it does not change any functionality.
Does @richcarl have any comments about this PR?
I also looked through some rendered functions and found some small irregularities that I'm not sure if you have seen or not:
erl_parse:tree/2
In the edoc html generate this function looks like this:
tree(Type::atom(),Data::term()) -> tree()
however, in the shell docs it looks like this:
-spec tree(atom(),term()) -> tree().
This because edoc for html reads the variable names and inserts them
into the spec. Information is lost when this is not done as the reader
does not know which argument is Type and which is Data in the text.
-spec tree(atom(), term()) -> tree().
For special purposes only. Creates an abstract syntax tree node
with type tag *Type* and associated data *Data*.
See also
See also data/1.
See also is_tree/1.
See also type/1.
These should probably be grouped into one:
See also data/1, is_tree/1, type/1
as that is done both in the edoc html and shell_docs.
Problem in shell_docs
Seems like shell_docs does not render <pre> correctly, or rather
it does not insert a newline before itself so that things like merl
look like this:
Quick start
To enable the full power of Merl, your module needs to include the
Merl header file:
-include_lib("syntax_tools/include/merl.hrl").
Then, you can use the ?Q(Text) macros in your code to create
ASTs or match on existing ASTs. For example:
Tuple = ?Q("{foo, 42}"),
?Q("{foo, _@Number}") = Tuple,
Call = ?Q("foo:bar(_@Number)")
Calling merl:print(Call) will then print the following code:
foo:bar(42)
The ?Q macros turn the quoted code fragments into ASTs, and
I'll take a look at fixing that and try not to break any of the erl_docgen renderings.
|
Thanks for the review, @garazdawi! I think it makes the most sense to start with the spec/type changes. I'm sending a separate PR shortly, to make it easy to review. Then, I'll rebase this as appropriate. |
erszcz
force-pushed
the
edoc-chunk-support
branch
from
fc009ab
to
f1c92fa
Compare
erszcz
changed the title
erszcz
force-pushed
the
edoc-chunk-support
branch
from
700bf56
to
eafc139
Compare
|
With regard to -spec tree(atom()) -> tree().
tree(Type) ->
tree(Type, []).
-spec tree(atom(), term()) -> tree().
tree(Type, Data) ->
...We get properly named types in It does not yet support 'bounded_fun' syntax tree nodes ( |
Done in the newest batch of changes. Moreover, arg names are now properly added to |
|
Would you mind rebasing on top of latest master to get the #2914 changes removed from this PR? |
|
@garazdawi Sure, I should be able to rebase this tomorrow. I also have some WIP on the documentation for the escript - I'll attach on top of the rebase. |
|
Great, let me know when you are ready for me to review the PR again. |
erszcz
force-pushed
the
edoc-chunk-support
branch
2 times, most recently
from
e10d3cb
to
db33aff
Compare
|
I've rebased this on top of the current master. I think I've addressed all points of the first review - please let me know if there's something missing. The current status is as follows:
TODOs:
|
|
Hello, I did Is that because they have both a |
I think it is fine that they do not match 100%. This is an oddity in the way that edoc behaves, but I don't think it is worth forcing users to rewrite their specs. |
I think it is better to do this in another PR. |
There was a problem hiding this comment.
I did a quick look to see if it would be easy to add support in the make system to use edoc instead of erl_docgen to build the doc chunks for Erlang/OTP and it turns out that it is not. So I think we will leave that for later.
My only remaining question from the original review is whether we should document the edoc_doclet_chunks and edoc_layout_chunks modules? The way it is implemented now it is only possible to generate chunks via the script without relying on undocumented API's.
erszcz
force-pushed
the
edoc-chunk-support
branch
from
d8b83b9
to
bc0d6d4
Compare
|
Status update:
Done.
Indeed, that turns out to be the reason. It seems that in such a case EDoc doesn't pass the -spec attr form down to the layout :| I might be able to find out why and fix it in EDoc core, but for now a warning will be printed that just |
In order to gather all info about a type that EEP-48 requires,
we refer to EDoc #tag{} records.
These records do not contain the type name at the top level,
so this lookup was done just by line number.
When a header file is included into another file,
it might happen that the position information of two distinct type
definitions is the same - they are still different types,
but the lookup by line will return the wrong one.
This lead to the following error (on kernel/src/disk_log.erl):
name: dlog_size
arity: 0
type attr: {attribute,82,type,{file_error,{type,82,term,[]},[]}}
edoc: error in layout 'edoc_layout_chunks': {'EXIT',
{{badmatch,{file_error,[]}},
[{edoc_layout_chunks,meta_type_sig,4,
[{file,"edoc_layout_chunks.erl"},
{line,170}]},
This change introduces a more sophisticated and robust lookup,
which fixes the problem of types defined in include files.
- Fix unused variables and lookup functions - Remove edoc-orig-tag - Remove name attribute from <a> elements - Be less verbose if not debugging
Co-authored-by: Lukas Larsson <[email protected]>
erszcz
force-pushed
the
edoc-chunk-support
branch
from
ea0046d
to
52dc947
Compare
This change also prevents the 'multiple @SPEC tag' error.
|
github actions doc build has started to fail with the latest changes, do you know why? |
|
ce77e68 and c5581d2 introduce the priority of attrs over tags. However, as this changes EDoc core, it's not limited to chunk generation, but all paths, so static HTML or Specs were missing in OTP docs after these changes - I'm now troubleshooting some types which used to be exported, but currently are not. |
erszcz
force-pushed
the
edoc-chunk-support
branch
from
93c34bb
to
52dc947
Compare
|
I’ve reverted the PR to the last commit passing all checks. I’ll carry on in a separate branch. |
Apparently, these are not handled properly by EDoc, therefore types mentioned in them aren't properly exported to docs.
|
The latest commits invert the priorities of tags and attributes. This priority inversion means that if a module has both a In case of specs, the problem of consecutive I finally think there's nothing more to add to this PR, unless some bugs pop up. @garazdawi @richcarl what do you think? |
I agree. |
|
Merged! Thanks for all your work! |
|
Woohoo! And thank you for the reviews! |
This PR enables EDoc to output EEP-48 style doc chunks. The work leading to this PR was so far discussed and documented at erlef/documentation-wg#4, whereas developed at https://github.com/erszcz/edoc.
I tried to address all points raised in the discussions so far:
.beamfiles.erl_docgenchunks as possible (with the caveat of type links described below).@specand@typetags are rewritten to-specand-typeattributes, respectively.We discussed the type link caveat with @garazdawi -
erl_docgenchunk links to types do not use type arities. This means links tot/0andt/1have the same form. EDoc chunks do not follow this principle, since it seemed more robust to allow linking to arbitrary types. Until this is fixed forerl_docgenchunks, measures have to be taken in 3rd party tools (e.g. ExDoc) to watch for this difference.Some features stemming from EDoc design and implementation, but not necessarily intuitive, are not addressed in this PR:
-specattributes, but type doc comments must follow-typeattributes.@sinceand@deprecatedtags are supported on functions, but are not supported on types or callbacks.This PR also adds
test/eep48_SUITE.erlwhich tests the majority of the newly added functionality, as well as some of the above points which are currently not supported (these tests are skipped by default).