Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new ruby parser that uses Prism #1144

Merged
merged 14 commits into from
Jul 31, 2024
Merged

Add new ruby parser that uses Prism #1144

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
61e7613
Add a new ruby parser RDoc::Parser::PrismRuby
tompng Jul 23, 2024
76d7786
Add a new ruby parser testcase independent from parser's internal imp…
tompng Jul 23, 2024
021d547
unknown meta method
tompng Jul 28, 2024
c44bd1f
Use MethodSignatureVisitor only to scan params, block_params and call…
tompng Jul 28, 2024
83b0fde
Add calls_super test
tompng Jul 28, 2024
aca20e1
Drop ruby 2.6. Prism requires ruby >= 2.7
tompng Jul 28, 2024
ec8ba9e
Remove duplicated documentation comment from prism_ruby.rb
tompng Jul 28, 2024
6531724
Add test for wrong argument passed to metaprogramming method
tompng Jul 28, 2024
f8a7c8f
Rename visit_call_[DSL_METHOD_NAME] to make it distinguishable from v…
tompng Jul 29, 2024
c6341ec
Method receiver switch of true/false/nil to a case statement
tompng Jul 29, 2024
00e164d
Extract common part of add_method(by def keyword) and add meta_commen…
tompng Jul 29, 2024
24ce586
Reuse consecutive comments array when collecting comments
tompng Jul 29, 2024
0a8634c
Simplify DSL call_node handling
tompng Jul 31, 2024
1ce2634
Refactor extracting method visibility arguments
tompng Jul 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions lib/rdoc/markup/pre_process.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,15 +97,18 @@ def initialize(input_file_name, include_path)
# RDoc::CodeObject#metadata for details.

def handle text, code_object = nil, &block
first_line = 1
if RDoc::Comment === text then
comment = text
text = text.text
first_line = comment.line || 1
end

# regexp helper (square brackets for optional)
# $1 $2 $3 $4 $5
# [prefix][\]:directive:[spaces][param]newline
text = text.gsub(/^([ \t]*(?:#|\/?\*)?[ \t]*)(\\?):(\w+):([ \t]*)(.+)?(\r?\n|$)/) do
text = text.lines.map.with_index(first_line) do |line, num|
next line unless line =~ /\A([ \t]*(?:#|\/?\*)?[ \t]*)(\\?):([\w-]+):([ \t]*)(.+)?(\r?\n|$)/
# skip something like ':toto::'
next $& if $4.empty? and $5 and $5[0, 1] == ':'

Expand All @@ -120,8 +123,8 @@ def handle text, code_object = nil, &block
next "#{$1.strip}\n"
end

handle_directive $1, $3, $5, code_object, text.encoding, &block
end
handle_directive $1, $3, $5, code_object, text.encoding, num, &block
end.join

if comment then
comment.text = text
Expand All @@ -148,7 +151,7 @@ def handle text, code_object = nil, &block
# When 1.8.7 support is ditched prefix can be defaulted to ''

def handle_directive prefix, directive, param, code_object = nil,
encoding = nil
encoding = nil, line = nil
blankline = "#{prefix.strip}\n"
directive = directive.downcase

Expand Down Expand Up @@ -220,11 +223,11 @@ def handle_directive prefix, directive, param, code_object = nil,
# remove parameter &block
code_object.params = code_object.params.sub(/,?\s*&\w+/, '') if code_object.params

code_object.block_params = param
code_object.block_params = param || ''

blankline
else
result = yield directive, param if block_given?
result = yield directive, param, line if block_given?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are these changes for? Looks like they're fixing issues that'd also affect the default Ruby parser?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the default ruby parser, the third parameter line is just ignored.

The line where :method: is written is the source line number of ghost method.

1| class A
2|   ##
3|   # this is a ghost method `foo(x, y). Source location is line 4
4|   # :method: foo
5|   # :args: x, y
6| end

The original code is calculating line number by this code.

rdoc/lib/rdoc/parser/ruby.rb

Lines 1094 to 1095 in 1bb3bec

if (comment.text = comment.text.sub(/^# +:?method: *(\S*).*?\n/i, '')) && !!$~ then
line_no += $`.count("\n")

pre_process.rb parses comment directives, parse_comment also parses some comment directives.
To avoid re-implementing these kind of thing, I changed PreProcess#handle to also provide line_number for each directive.

This is not the best way. pre_process has problem and I think it should be fixed in the future. pre_process needs code_object already created, but we need to parse the comment first to identify code_object type which can be either attribute or method.


case result
when nil then
Expand Down
Loading