rustdoc-search: search types by higher-order functions

[notriddle]

This feature extends rustdoc with syntax and search index information for searching function pointers and closures (Higher-Order Functions, or HOF). Part of #60485

This PR adds two syntaxes: a high-level one for finding any kind of HOF, and a direct implementation of the parenthesized path syntax that Rust itself uses.

Preview pages

Query	Results
option<T>, (fnonce (T) -> bool) -> option<T>	Option::filter
option<T>, (T -> bool) -> option<T>	Option::filter

Updated chapter of the book: https://notriddle.com/rustdoc-html-demo-9/search-hof/rustdoc/read-documentation/search.html

Motivation

When type-based search was first landed, it was directly described as incomplete.

Filling out the missing functionality is going to mean adding support for more of Rust's type expression syntax, such as references, raw pointers, function pointers, and closures. This PR adds function pointers and closures.

There's been demand for something "like Hoogle, but for Rust" expressed a few times 1 2 3 4. Some of them just don't realize what functionality already exists (Duration -> u64 already works), but a lot of them specifically want to search for higher-order functions like option combinators.

Guide-level explanation (from the Rustdoc book)

To search for a function that accepts a function as a parameter, like Iterator::all, wrap the nested signature in parenthesis, as in Iterator<T>, (T -> bool) -> bool. You can also search for a specific closure trait, such as Iterator<T>, (FnMut(T) -> bool) -> bool, but you need to know which one you want.

Reference-level description (also from the Rustdoc book)

Primitives with Special Syntax

Shorthand	Explicit names
Before this PR
[]	primitive:slice and/or primitive:array
[T]	primitive:slice<T> and/or primitive:array<T>
!	primitive:never
()	primitive:unit and/or primitive:tuple
(T)	T
(T,)	primitive:tuple<T>
After this PR
(T, U -> V, W)	fn(T, U) -> (V, W), Fn, FnMut, and FnOnce

The -> operator has lower precedence than comma. If it's not wrapped in brackets, it delimits the return value for the function being searched for. To search for functions that take functions as parameters, use parenthesis.

Search query grammar

ident = *(ALPHA / DIGIT / "_")
path = ident *(DOUBLE-COLON ident) [BANG]
slice-like = OPEN-SQUARE-BRACKET [ nonempty-arg-list ] CLOSE-SQUARE-BRACKET
tuple-like = OPEN-PAREN [ nonempty-arg-list ] CLOSE-PAREN
arg = [type-filter *WS COLON *WS] (path [generics] / slice-like / tuple-like)
type-sep = COMMA/WS *(COMMA/WS)
nonempty-arg-list = *(type-sep) arg *(type-sep arg) *(type-sep) [ return-args ]
generic-arg-list = *(type-sep) arg [ EQUAL arg ] *(type-sep arg [ EQUAL arg ]) *(type-sep)
normal-generics = OPEN-ANGLE-BRACKET [ generic-arg-list ] *(type-sep)
            CLOSE-ANGLE-BRACKET
fn-like-generics = OPEN-PAREN [ nonempty-arg-list ] CLOSE-PAREN [ RETURN-ARROW arg ]
generics = normal-generics / fn-like-generics
return-args = RETURN-ARROW *(type-sep) nonempty-arg-list

exact-search = [type-filter *WS COLON] [ RETURN-ARROW ] *WS QUOTE ident QUOTE [ generics ]
type-search = [ nonempty-arg-list ]

query = *WS (exact-search / type-search) *WS

; unchanged parts of the grammar, like the full list of type filters, are omitted

Future direction

The remaining type expression grammar

As described in #118194, this is another step in the type expression grammar: BareFunction, and the function-like mode of TypePath, are now supported.

• RawPointerType and ReferenceType actually are a priority.
• ImplTraitType and TraitObjectType (and ImplTraitTypeOneBound and TraitObjectTypeOneBound) aren't as much of a priority, since they desugar pretty easily.

Search subtyping and traits

This is the other major factor that makes it less useful than it should be.

• iterator<result<t>> -> result<t> doesn't find Result::from_iter. You have to search intoiterator<result<t>> -> result<t>. Nobody's going to search for IntoIterator unless they basically already know about it and don't need the search engine anyway.

• Iterator combinators are usually structs that happen to implement Iterator, like std::iter::Map.

To solve these cases, it needs to look at trait implementations, knowing that Iterator is a "subtype of" IntoIterator, and Map is a "subtype of" Iterator, so iterator -> result is a subtype of intoiterator -> result and iterator<t>, (t -> u) -> iterator<u> is a subtype of iterator<t>, (t -> u) -> map<t -> u>.

[rustbot]

Some changes occurred in HTML/CSS/JS.

cc @GuillaumeGomez, @jsha

[GuillaumeGomez]

The syntax complexity for the search is to be considered here. I'm personally in favour, but I wonder if we should keep pushing more advanced search features or not. I'll add it to the rustdoc team meeting agenda so it can be discussed.

[notriddle]

The syntax complexity for the search is to be considered here. I'm personally in favour, but I wonder if we should keep pushing more advanced search features or not. I'll add it to the rustdoc team meeting agenda so it can be discussed.

That doesn't seem avoidable if I want these two things:

• Everyone wants the function-signature-search pages to show the signature.
• I want the visible signature to act as a search tutorial, which means you should be able to copy and paste a type signature from a result and search for it.

[GuillaumeGomez]

It was discussed in today's rustdoc meeting and approved by the team, so let's move forward with this. Thanks!

@bors r+ rollup

[bors]

📌 Commit 84d7a2c has been approved by GuillaumeGomez

It is now in the queue for this repository.

[workingjubilee]

Broke in #122350. Please rebase and fix things for tidy's preferences about file length.

@bors r-

[notriddle]

@bors r=GuillaumeGomez

[bors]

📌 Commit 7b92655 has been approved by GuillaumeGomez

It is now in the queue for this repository.