Rustdoc search by type #23289

mihneadb · 2015-03-11T21:57:57Z

This adds search by type (for functions/methods) support to Rustdoc. Target issue is at rust-lang/rfcs#658.

I've described my approach here: rust-lang/rfcs#658 (comment). I'll copy the text in here as well:

Hi, it took me longer than I wished, but I have implemented this in a not-too-complex way that I think can be extended to support more complex features (like the ones mentioned here).

The idea is to generate a JSON representation of the types of methods/functions in the existing index, and then make the JS understand when it should look by type (and not by name).

I tried to come up with a JSON representation that can be extended to support generics, bounds, ref/mut annotations and so on. Here are a few samples:

Function:

fn to_uppercase(c: char) -> char

{
    "inputs": [
        {"name": "char"}
    ],
    "output": {
        "name": "char",
    }
}

Method (implemented or defined in trait):

// in struct Vec
// self is considered an argument as well
fn capacity(&self) -> usize

{
    "inputs": [
        {"name": "vec"}
    ],
    "output": {
        "name": "usize"
    }
}

This simple format can be extended by adding more fields, like generic: bool, a bounds mapping and so on.

I have a working implementation in master...mihneadb:rustdoc-search-by-type. You can check out a live demo here.

The feature list is not that long:

search by types (you can use generics as well, as long as you use the exact name - e.g. vec,t ->)
order of arguments does not matter
self is took into account as well (e.g. search for vec -> usize)
does not use "complex" annotations (e.g. you don't search for &char -> char but for char -> char)

My goal is to get a working, minimal "base" merged so that others can build upon it. How should I proceed? Do I open a PR (badly in need of code review since this is my first non "hello world"-ish rust code)?

rust-highfive · 2015-03-11T21:58:04Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @pcwalton (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. The way Github handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see CONTRIBUTING.md for more information.

mihneadb · 2015-03-11T21:58:13Z

cc @steveklabnik

steveklabnik · 2015-03-11T22:01:37Z

I am not great with Rustdoc internals, but really want this as a feature. 🎊

mihneadb · 2015-03-11T22:05:10Z

Addressing the make tidy issues.

jdm · 2015-03-11T22:30:54Z

This is totally excellent! 🎃

killercup · 2015-03-11T22:52:12Z

This looks really good!

A next step might be fuzzy matching on types. E.g. -> Result or File ->.

(Your demo page took forever to load, by the way. 5 1/2 min—I'm not kidding! Is that server on Mars? You might also want enable gzip; the raw search index is 3.5MB.)

mihneadb · 2015-03-12T08:52:57Z

@killercup definitely! I'd rather do it in a further iteration if that's OK.

Regarding the server, I sometimes wonder the same thing. Sorry about that! From my tests the load time was OK.

Since the docs can be accessed in many ways (i.e. not necessarily by a server that can do gzip), I'm thinking of actually gzipping the index from the Rust code and have it decompressed in JS. What do you think?

killercup · 2015-03-12T09:04:18Z

@mihneadb these were just queries I wanted to do that don't work right now. Launch this as is, and we can add more stuff later.

I have no idea how performant compressing this stuff in JS is. pako looks like a nice library (the deflate part is 26k minified). You might want to test how well that works. Don't forget that these docs are also served locally; I don't know how to load a compressed file so that it can be read in JS without using AJAX requests.

If you are concerned about the file size, it might help to restructure the JSON output a bit. E.g. change

{
  "inputs": [
    {"name": "char"},
    {"name": "i32"}
  ],
  "output": {
    "name": "char",
  }
}

to:

[
  ["char", "i32"],
  "char"
]

mihneadb · 2015-03-12T10:07:59Z

@killercup for the record, you can search for no input / no output (e.g. vec -> and -> vec), but it's not fuzzy. Thanks for the suggestions!

alexcrichton · 2015-03-13T18:57:24Z

src/librustdoc/html/render.rs

+                            }
+                        } else {
+                            None
+                        };


I believe our typical form of indentation doesn't indent the body of the if over one extra (e.g. this should be tabbed left one).

I also think that this may be better expressed as self.parent_stack.first().and_then(|..| { ... }) as it's already got the Option there.

mihneadb · 2015-03-13T20:47:20Z

@alexcrichton thanks a lot for the input, I learned a lot!

mihneadb · 2015-03-13T20:47:56Z

src/librustdoc/html/render.rs

+    /// Formats type as {name: $name}.
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        try!(write!(f, "{{"));
+        try!(write!(f, "\"name\":\"{}\"", self.name.clone().unwrap_or("UNKNOWN".to_string())));


@alexcrichton any way to make this shorter/without using clone?

Perhaps! I was thinking though that we may want to use null here instead of UNKNOWN (as searching for UNKNOWN probably isn't what you want).

I wonder, could entries with None be omitted entirely from the search index?

They shouldn't be omitted entirely from the search index, since you might still be able to search them by name. However, the search_type of such items should be serialized as null (JS) alltogether. On it!

jstasiak · 2015-03-13T21:06:40Z

+1, I'm new to Rust and this is a very desired feature

alexcrichton · 2015-03-13T21:40:12Z

This is looking really great, nice work! I can think of a number of cases that aren't quite covered by this, but I feel like this is a great improvement regardless and it can always be iterated on so I'm fine landing with a few known cases where it may not work :)

mihneadb · 2015-03-13T21:42:48Z

Thanks for the kind words, I totally agree that it can be improved (I want to try adding some things in future iterations, after v0 gets landed).

mihneadb · 2015-03-13T22:54:47Z

@alexcrichton Please take another look. The code is now ignoring types it can't handle and it just writes null for the search_type of the corresponding entry. I think the PR is ready now.

alexcrichton · 2015-03-13T23:37:40Z

src/librustdoc/html/render.rs

+        try!(write!(f, "{{"));
+        // Wrapping struct fmt should never call us when self.name is None,
+        // but just to be safe we write `null` in that case.
+        try!(write!(f, "\"name\":\"{}\"", self.name.as_ref().unwrap_or(&"null".to_string())));


If the name is None, I think this will output "null" instead of null (which may confuse the JS)

steveklabnik · 2015-03-14T20:21:23Z

@bors: r=alexcrichton

bors · 2015-03-14T20:21:25Z

~~@bors r=alexcrichton 7b7b938~~

bors · 2015-03-14T22:07:26Z

⌛ Testing commit 7b7b938 with merge 30e1f9a...

This adds search by type (for functions/methods) support to Rustdoc. Target issue is at rust-lang/rfcs#658. I've described my approach here: rust-lang/rfcs#658 (comment). I'll copy the text in here as well: --- Hi, it took me longer than I wished, but I have implemented this in a not-too-complex way that I think can be extended to support more complex features (like the ones mentioned [here](#12866 (comment))). The idea is to generate a JSON representation of the types of methods/functions in the existing index, and then make the JS understand when it should look by type (and not by name). I tried to come up with a JSON representation that can be extended to support generics, bounds, ref/mut annotations and so on. Here are a few samples: Function: ```rust fn to_uppercase(c: char) -> char ``` ```json { "inputs": [ {"name": "char"} ], "output": { "name": "char", } } ``` Method (implemented or defined in trait): ```rust // in struct Vec // self is considered an argument as well fn capacity(&self) -> usize ``` ```json { "inputs": [ {"name": "vec"} ], "output": { "name": "usize" } } ``` This simple format can be extended by adding more fields, like `generic: bool`, a `bounds` mapping and so on. I have a working implementation in master...mihneadb:rustdoc-search-by-type. You can check out a live demo [here](http://data.mihneadb.net/doc/std/index.html?search=charext%20-%3E%20char). ![screenshot from 2015-02-28 00 54 00](https://cloud.githubusercontent.com/assets/643127/6422722/7e5374ee-bee4-11e4-99a6-9aac3c9d5068.png) The feature list is not that long: - search by types (you *can* use generics as well, as long as you use the exact name - e.g. [`vec,t -> `](http://data.mihneadb.net/doc/std/index.html?search=vec%2C%20t%20-%3E)) - order of arguments does not matter - `self` is took into account as well (e.g. search for `vec -> usize`) - does not use "complex" annotations (e.g. you don't search for `&char -> char` but for `char -> char`) My goal is to get a working, minimal "base" merged so that others can build upon it. How should I proceed? Do I open a PR (badly in need of code review since this is my first non "hello world"-ish rust code)? ---

bors · 2015-03-15T00:39:53Z

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-x-android-t, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-win-32-nopt-t, auto-win-32-opt, auto-win-64-nopt-t, auto-win-64-opt

@steveklabnik

…bnik I noticed this feature added in rust-lang#23289 was missing from the `Search tricks`. Thanks! r? @steveklabnik

@steveklabnik

…bnik I noticed this feature added in rust-lang#23289 was missing from the `Search tricks`. Thanks! r? @steveklabnik

@alexcrichton

…crichton In #23289, I did not include the type information for searching for orphan methods (methods that are defined in a module and implemented in another - doing this causes rustdoc sometimes to first find the impl before the actual type). This PR fixes this. By merging this, searches for `char -> bool` will also find `is_alphabetic`, for example (which otherwise was only found for `charext -> bool`). cc @alexcrichton

…e-array, r=GuillaumeGomez rustdoc: search for slices and arrays by type with `[]` This feature extends rustdoc to support the syntax that most users will naturally attempt to use to search for slices and arrays. Part of rust-lang#60485 Function signature searches already support arrays and slices. The explicit name `primitive:slice<u8>` and `primitive:array<u8>` can be used to match a slice or array of bytes, while square brackets `[u8]` will match either one. Empty square brackets, `[]`, will match any slice regardless of what it contains. Preview: * [`option -> []`](https://notriddle.com/rustdoc-demo-html-3/search-slice-array/std/index.html?search=option%20-%3E%20%5B%5D) * [`[u8] -> str`](https://notriddle.com/rustdoc-demo-html-3/search-slice-array/std/index.html?search=%5Bu8%5D%20-%3E%20str) * [`Box<[u8]> -> str`](https://notriddle.com/rustdoc-demo-html-3/search-slice-array/std/index.html?search=Box%3C%5Bu8%5D%3E%20-%3E%20str) Motivation: When type-based search was first landed, it was directly described as "incomplete". Here's [a comment] from the discussion thread: [a comment]: rust-lang#23289 (comment) > This is looking really great, nice work! I can think of a number of cases that aren't quite covered by this, but I feel like this is a great improvement regardless and it can always be iterated on so I'm fine landing with a few known cases where it may not work :) Filling out the missing functionality is going to mean adding support for more of Rust's [type expression] syntax, such as slices (in this PR), tuples, references, raw pointers, function pointers, and generics. [type expression]: https://doc.rust-lang.org/reference/types.html#type-expressions There does seem to be demand for this sort of thing, such as [this Discord message](https://discord.com/channels/442252698964721669/443150878111694848/1042145740065099796) expressing regret at rustdoc not supporting tuples in search queries.

…GuillaumeGomez rustdoc: search for tuples and unit by type with `()` This feature extends rustdoc to support the syntax that most users will naturally attempt to use to search for tuples. Part of rust-lang#60485 Function signature searches already support tuples and unit. The explicit name `primitive:tuple` and `primitive:unit` can be used to match a tuple or unit, while `()` will match either one. It also follows the direction set by the actual language for parens as a group, so `(u8,)` will only match a tuple, while `(u8)` will match a plain, unwrapped byte—thanks to loose search semantics, it will also match the tuple. ## Preview * [`option<t>, option<u> -> (t, u)`](<https://notriddle.com/rustdoc-html-demo-5/tuple-unit/std/index.html?search=option%3Ct%3E%2C option%3Cu%3E -%3E (t%2C u)>) * [`[t] -> (t,)`](<https://notriddle.com/rustdoc-html-demo-5/tuple-unit/std/index.html?search=[t] -%3E (t%2C)>) * [`(ipaddr,) -> socketaddr`](<https://notriddle.com/rustdoc-html-demo-5/tuple-unit/std/index.html?search=(ipaddr%2C) -%3E socketaddr>) ## Motivation When type-based search was first landed, it was directly [described as incomplete][a comment]. [a comment]: rust-lang#23289 (comment) Filling out the missing functionality is going to mean adding support for more of Rust's [type expression] syntax, such as tuples (in this PR), references, raw pointers, function pointers, and closures. [type expression]: https://doc.rust-lang.org/reference/types.html#type-expressions There does seem to be demand for this sort of thing, such as [this Discord message](https://discord.com/channels/442252698964721669/443150878111694848/1042145740065099796) expressing regret at rustdoc not supporting tuples in search queries. ## Reference description (from the Rustdoc book) <table> <thead> <tr> <th>Shorthand</th> <th>Explicit names</th> </tr> </thead> <tbody> <tr><td colspan="2">Before this PR</td></tr> <tr> <td><code>[]</code></td> <td><code>primitive:slice</code> and/or <code>primitive:array</code></td> </tr> <tr> <td><code>[T]</code></td> <td><code>primitive:slice<T></code> and/or <code>primitive:array<T></code></td> </tr> <tr> <td><code>!</code></td> <td><code>primitive:never</code></td> </tr> <tr><td colspan="2">After this PR</td></tr> <tr> <td><code>()</code></td> <td><code>primitive:unit</code> and/or <code>primitive:tuple</code></td> </tr> <tr> <td><code>(T)</code></td> <td><code>T</code></td> </tr> <tr> <td><code>(T,)</code></td> <td><code>primitive:tuple<T></code></td> </tr> </tbody> </table> A single type expression wrapped in parens is the same as that type expression, since parens act as the grouping operator. If they're empty, though, they will match both `unit` and `tuple`, and if there's more than one type (or a trailing or leading comma) it is the same as `primitive:tuple<...>`. However, since items can be left out of the query, `(T)` will still return results for types that match tuples, even though it also matches the type on its own. That is, `(u32)` matches `(u32,)` for the exact same reason that it also matches `Result<u32, Error>`. ## Future direction The [type expression grammar](https://doc.rust-lang.org/reference/types.html#type-expressions) from the Reference is given below: <pre><code>Syntax Type : TypeNoBounds | <a href="https://doc.rust-lang.org/reference/types/impl-trait.html">ImplTraitType</a> | <a href="https://doc.rust-lang.org/reference/types/trait-object.html">TraitObjectType</a> <br> TypeNoBounds : <a href="https://doc.rust-lang.org/reference/types.html#parenthesized-types">ParenthesizedType</a> | <a href="https://doc.rust-lang.org/reference/types/impl-trait.html">ImplTraitTypeOneBound</a> | <a href="https://doc.rust-lang.org/reference/types/trait-object.html">TraitObjectTypeOneBound</a> | <a href="https://doc.rust-lang.org/reference/paths.html#paths-in-types">TypePath</a> | <a href="https://doc.rust-lang.org/reference/types/tuple.html#tuple-types">TupleType</a> | <a href="https://doc.rust-lang.org/reference/types/never.html">NeverType</a> | <a href="https://doc.rust-lang.org/reference/types/pointer.html#raw-pointers-const-and-mut">RawPointerType</a> | <a href="https://doc.rust-lang.org/reference/types/pointer.html#shared-references-">ReferenceType</a> | <a href="https://doc.rust-lang.org/reference/types/array.html">ArrayType</a> | <a href="https://doc.rust-lang.org/reference/types/slice.html">SliceType</a> | <a href="https://doc.rust-lang.org/reference/types/inferred.html">InferredType</a> | <a href="https://doc.rust-lang.org/reference/paths.html#qualified-paths">QualifiedPathInType</a> | <a href="https://doc.rust-lang.org/reference/types/function-pointer.html">BareFunctionType</a> | <a href="https://doc.rust-lang.org/reference/macros.html#macro-invocation">MacroInvocation</a> </code></pre> ImplTraitType and TraitObjectType (and ImplTraitTypeOneBound and TraitObjectTypeOneBound) are not yet implemented. They would mostly desugar to `trait:`, similarly to how `!` desugars to `primitive:never`. ParenthesizedType and TuplePath are added in this PR. TypePath is already implemented (except const generics, which is not planned, and function-like trait syntax, which is planned as part of closure support). NeverType is already implemented. RawPointerType and ReferenceType require parsing and fixes to the search index to store this information, but otherwise their behavior seems simple enough. Just like tuples and slices, `&T` would be equivalent to `primitive:reference<T>`, `&mut T` would be equivalent to `primitive:reference<keyword:mut, T>`, `*T` would be equivalent to `primitive:pointer<T>`, `*mut T` would be equivalent to `primitive:pointer<keyword:mut, T>`, and `*const T` would be equivalent to `primitive:pointer<keyword:const, T>`. Lifetime generics support is not planned, because lifetime subtyping seems too complicated. ArrayType is subsumed by SliceType right now. Implementing const generics is not planned, because it seems like it would require a lot of implementation complexity for not much gain. InferredType isn't really covered right now. Its semantics in a search context are not obvious. QualifiedPathInType is not implemented, and it is not planned. I would need a use case to justify it, and act as a guide for what the exact semantics should be. BareFunctionType is not implemented. Along with function-like trait syntax, which is formally considered a TypePath, it's the biggest missing feature to be able to do structured searches over generic APIs like `Option`. MacroInvocation is not parsed (macro names are, but they don't mean the same thing here at all). Those are gone by the time Rustdoc sees the source code.

Rollup merge of rust-lang#118194 - notriddle:notriddle/tuple-unit, r=GuillaumeGomez rustdoc: search for tuples and unit by type with `()` This feature extends rustdoc to support the syntax that most users will naturally attempt to use to search for tuples. Part of rust-lang#60485 Function signature searches already support tuples and unit. The explicit name `primitive:tuple` and `primitive:unit` can be used to match a tuple or unit, while `()` will match either one. It also follows the direction set by the actual language for parens as a group, so `(u8,)` will only match a tuple, while `(u8)` will match a plain, unwrapped byte—thanks to loose search semantics, it will also match the tuple. ## Preview * [`option<t>, option<u> -> (t, u)`](<https://notriddle.com/rustdoc-html-demo-5/tuple-unit/std/index.html?search=option%3Ct%3E%2C option%3Cu%3E -%3E (t%2C u)>) * [`[t] -> (t,)`](<https://notriddle.com/rustdoc-html-demo-5/tuple-unit/std/index.html?search=[t] -%3E (t%2C)>) * [`(ipaddr,) -> socketaddr`](<https://notriddle.com/rustdoc-html-demo-5/tuple-unit/std/index.html?search=(ipaddr%2C) -%3E socketaddr>) ## Motivation When type-based search was first landed, it was directly [described as incomplete][a comment]. [a comment]: rust-lang#23289 (comment) Filling out the missing functionality is going to mean adding support for more of Rust's [type expression] syntax, such as tuples (in this PR), references, raw pointers, function pointers, and closures. [type expression]: https://doc.rust-lang.org/reference/types.html#type-expressions There does seem to be demand for this sort of thing, such as [this Discord message](https://discord.com/channels/442252698964721669/443150878111694848/1042145740065099796) expressing regret at rustdoc not supporting tuples in search queries. ## Reference description (from the Rustdoc book) <table> <thead> <tr> <th>Shorthand</th> <th>Explicit names</th> </tr> </thead> <tbody> <tr><td colspan="2">Before this PR</td></tr> <tr> <td><code>[]</code></td> <td><code>primitive:slice</code> and/or <code>primitive:array</code></td> </tr> <tr> <td><code>[T]</code></td> <td><code>primitive:slice<T></code> and/or <code>primitive:array<T></code></td> </tr> <tr> <td><code>!</code></td> <td><code>primitive:never</code></td> </tr> <tr><td colspan="2">After this PR</td></tr> <tr> <td><code>()</code></td> <td><code>primitive:unit</code> and/or <code>primitive:tuple</code></td> </tr> <tr> <td><code>(T)</code></td> <td><code>T</code></td> </tr> <tr> <td><code>(T,)</code></td> <td><code>primitive:tuple<T></code></td> </tr> </tbody> </table> A single type expression wrapped in parens is the same as that type expression, since parens act as the grouping operator. If they're empty, though, they will match both `unit` and `tuple`, and if there's more than one type (or a trailing or leading comma) it is the same as `primitive:tuple<...>`. However, since items can be left out of the query, `(T)` will still return results for types that match tuples, even though it also matches the type on its own. That is, `(u32)` matches `(u32,)` for the exact same reason that it also matches `Result<u32, Error>`. ## Future direction The [type expression grammar](https://doc.rust-lang.org/reference/types.html#type-expressions) from the Reference is given below: <pre><code>Syntax Type : TypeNoBounds | <a href="https://doc.rust-lang.org/reference/types/impl-trait.html">ImplTraitType</a> | <a href="https://doc.rust-lang.org/reference/types/trait-object.html">TraitObjectType</a> <br> TypeNoBounds : <a href="https://doc.rust-lang.org/reference/types.html#parenthesized-types">ParenthesizedType</a> | <a href="https://doc.rust-lang.org/reference/types/impl-trait.html">ImplTraitTypeOneBound</a> | <a href="https://doc.rust-lang.org/reference/types/trait-object.html">TraitObjectTypeOneBound</a> | <a href="https://doc.rust-lang.org/reference/paths.html#paths-in-types">TypePath</a> | <a href="https://doc.rust-lang.org/reference/types/tuple.html#tuple-types">TupleType</a> | <a href="https://doc.rust-lang.org/reference/types/never.html">NeverType</a> | <a href="https://doc.rust-lang.org/reference/types/pointer.html#raw-pointers-const-and-mut">RawPointerType</a> | <a href="https://doc.rust-lang.org/reference/types/pointer.html#shared-references-">ReferenceType</a> | <a href="https://doc.rust-lang.org/reference/types/array.html">ArrayType</a> | <a href="https://doc.rust-lang.org/reference/types/slice.html">SliceType</a> | <a href="https://doc.rust-lang.org/reference/types/inferred.html">InferredType</a> | <a href="https://doc.rust-lang.org/reference/paths.html#qualified-paths">QualifiedPathInType</a> | <a href="https://doc.rust-lang.org/reference/types/function-pointer.html">BareFunctionType</a> | <a href="https://doc.rust-lang.org/reference/macros.html#macro-invocation">MacroInvocation</a> </code></pre> ImplTraitType and TraitObjectType (and ImplTraitTypeOneBound and TraitObjectTypeOneBound) are not yet implemented. They would mostly desugar to `trait:`, similarly to how `!` desugars to `primitive:never`. ParenthesizedType and TuplePath are added in this PR. TypePath is already implemented (except const generics, which is not planned, and function-like trait syntax, which is planned as part of closure support). NeverType is already implemented. RawPointerType and ReferenceType require parsing and fixes to the search index to store this information, but otherwise their behavior seems simple enough. Just like tuples and slices, `&T` would be equivalent to `primitive:reference<T>`, `&mut T` would be equivalent to `primitive:reference<keyword:mut, T>`, `*T` would be equivalent to `primitive:pointer<T>`, `*mut T` would be equivalent to `primitive:pointer<keyword:mut, T>`, and `*const T` would be equivalent to `primitive:pointer<keyword:const, T>`. Lifetime generics support is not planned, because lifetime subtyping seems too complicated. ArrayType is subsumed by SliceType right now. Implementing const generics is not planned, because it seems like it would require a lot of implementation complexity for not much gain. InferredType isn't really covered right now. Its semantics in a search context are not obvious. QualifiedPathInType is not implemented, and it is not planned. I would need a use case to justify it, and act as a guide for what the exact semantics should be. BareFunctionType is not implemented. Along with function-like trait syntax, which is formally considered a TypePath, it's the biggest missing feature to be able to do structured searches over generic APIs like `Option`. MacroInvocation is not parsed (macro names are, but they don't mean the same thing here at all). Those are gone by the time Rustdoc sees the source code.

…-hof, r=GuillaumeGomez rustdoc-search: search types by higher-order functions This feature extends rustdoc with syntax and search index information for searching function pointers and closures (Higher-Order Functions, or HOF). Part of rust-lang#60485 This PR adds two syntaxes: a high-level one for finding any kind of HOF, and a direct implementation of the parenthesized path syntax that Rust itself uses. ## Preview pages | Query | Results | |-------|---------| | [`option<T>, (fnonce (T) -> bool) -> option<T>`][optionfilter] | `Option::filter` | | [`option<T>, (T -> bool) -> option<T>`][optionfilter2] | `Option::filter` | Updated chapter of the book: https://notriddle.com/rustdoc-html-demo-9/search-hof/rustdoc/read-documentation/search.html [optionfilter]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=option<T>%2C+(fnonce+(T)+->+bool)+->+option<T>&filter-crate=std [optionfilter2]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=option<T>%2C+(T+->+bool)+->+option<T>&filter-crate=std ## Motivation When type-based search was first landed, it was directly [described as incomplete][a comment]. [a comment]: rust-lang#23289 (comment) Filling out the missing functionality is going to mean adding support for more of Rust's [type expression] syntax, such as references, raw pointers, function pointers, and closures. This PR adds function pointers and closures. [type expression]: https://doc.rust-lang.org/reference/types.html#type-expressions There's been demand for something "like Hoogle, but for Rust" expressed a few times [1](https://www.reddit.com/r/rust/comments/y8sbid/is_there_a_website_like_haskells_hoogle_for_rust/) [2](https://users.rust-lang.org/t/rust-equivalent-of-haskells-hoogle/102280) [3](https://internals.rust-lang.org/t/std-library-inclusion-policy/6852/2) [4](https://discord.com/channels/442252698964721669/448238009733742612/1109502307495858216). Some of them just don't realize what functionality already exists ([`Duration -> u64`](https://doc.rust-lang.org/nightly/std/?search=duration%20-%3E%20u64) already works), but a lot of them specifically want to search for higher-order functions like option combinators. ## Guide-level explanation (from the Rustdoc book) To search for a function that accepts a function as a parameter, like `Iterator::all`, wrap the nested signature in parenthesis, as in [`Iterator<T>, (T -> bool) -> bool`][iterator-all]. You can also search for a specific closure trait, such as `Iterator<T>, (FnMut(T) -> bool) -> bool`, but you need to know which one you want. [iterator-all]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=Iterator<T>%2C+(T+->+bool)+->+bool&filter-crate=std ## Reference-level description (also from the Rustdoc book) ### Primitives with Special Syntax <table> <thead> <tr> <th>Shorthand</th> <th>Explicit names</th> </tr> </thead> <tbody> <tr><td colspan="2">Before this PR</td></tr> <tr> <td><code>[]</code></td> <td><code>primitive:slice</code> and/or <code>primitive:array</code></td> </tr> <tr> <td><code>[T]</code></td> <td><code>primitive:slice<T></code> and/or <code>primitive:array<T></code></td> </tr> <tr> <td><code>!</code></td> <td><code>primitive:never</code></td> </tr> <tr> <td><code>()</code></td> <td><code>primitive:unit</code> and/or <code>primitive:tuple</code></td> </tr> <tr> <td><code>(T)</code></td> <td><code>T</code></td> </tr> <tr> <td><code>(T,)</code></td> <td><code>primitive:tuple<T></code></td> </tr> <tr><td colspan="2">After this PR</td></tr> <tr> <td><code>(T, U -> V, W)</code></td> <td><code>fn(T, U) -> (V, W)</code>, Fn, FnMut, and FnOnce</td> </tr> </tbody> </table> The `->` operator has lower precedence than comma. If it's not wrapped in brackets, it delimits the return value for the function being searched for. To search for functions that take functions as parameters, use parenthesis. ### Search query grammar ```ebnf ident = *(ALPHA / DIGIT / "_") path = ident *(DOUBLE-COLON ident) [BANG] slice-like = OPEN-SQUARE-BRACKET [ nonempty-arg-list ] CLOSE-SQUARE-BRACKET tuple-like = OPEN-PAREN [ nonempty-arg-list ] CLOSE-PAREN arg = [type-filter *WS COLON *WS] (path [generics] / slice-like / tuple-like) type-sep = COMMA/WS *(COMMA/WS) nonempty-arg-list = *(type-sep) arg *(type-sep arg) *(type-sep) [ return-args ] generic-arg-list = *(type-sep) arg [ EQUAL arg ] *(type-sep arg [ EQUAL arg ]) *(type-sep) normal-generics = OPEN-ANGLE-BRACKET [ generic-arg-list ] *(type-sep) CLOSE-ANGLE-BRACKET fn-like-generics = OPEN-PAREN [ nonempty-arg-list ] CLOSE-PAREN [ RETURN-ARROW arg ] generics = normal-generics / fn-like-generics return-args = RETURN-ARROW *(type-sep) nonempty-arg-list exact-search = [type-filter *WS COLON] [ RETURN-ARROW ] *WS QUOTE ident QUOTE [ generics ] type-search = [ nonempty-arg-list ] query = *WS (exact-search / type-search) *WS ; unchanged parts of the grammar, like the full list of type filters, are omitted ``` ## Future direction ### The remaining type expression grammar As described in rust-lang#118194, this is another step in the type expression grammar: BareFunction, and the function-like mode of TypePath, are now supported. * RawPointerType and ReferenceType actually are a priority. * ImplTraitType and TraitObjectType (and ImplTraitTypeOneBound and TraitObjectTypeOneBound) aren't as much of a priority, since they desugar pretty easily. ### Search subtyping and traits This is the other major factor that makes it less useful than it should be. * `iterator<result<t>> -> result<t>` doesn't find `Result::from_iter`. You have to search [`intoiterator<result<t>> -> result<t>`](https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=intoiterator%3Cresult%3Ct%3E%3E%20-%3E%20result%3Ct%3E&filter-crate=std). Nobody's going to search for IntoIterator unless they basically already know about it and don't need the search engine anyway. * Iterator combinators are usually structs that happen to implement Iterator, like `std::iter::Map`. To solve these cases, it needs to look at trait implementations, knowing that Iterator is a "subtype of" IntoIterator, and Map is a "subtype of" Iterator, so `iterator -> result` is a subtype of `intoiterator -> result` and `iterator<t>, (t -> u) -> iterator<u>` is a subtype of [`iterator<t>, (t -> u) -> map<t -> u>`](https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=iterator%3Ct%3E%2C%20(t%20-%3E%20u)%20-%3E%20map%3Ct%20-%3E%20u%3E&filter-crate=std).

Rollup merge of rust-lang#119676 - notriddle:notriddle/rustdoc-search-hof, r=GuillaumeGomez rustdoc-search: search types by higher-order functions This feature extends rustdoc with syntax and search index information for searching function pointers and closures (Higher-Order Functions, or HOF). Part of rust-lang#60485 This PR adds two syntaxes: a high-level one for finding any kind of HOF, and a direct implementation of the parenthesized path syntax that Rust itself uses. ## Preview pages | Query | Results | |-------|---------| | [`option<T>, (fnonce (T) -> bool) -> option<T>`][optionfilter] | `Option::filter` | | [`option<T>, (T -> bool) -> option<T>`][optionfilter2] | `Option::filter` | Updated chapter of the book: https://notriddle.com/rustdoc-html-demo-9/search-hof/rustdoc/read-documentation/search.html [optionfilter]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=option<T>%2C+(fnonce+(T)+->+bool)+->+option<T>&filter-crate=std [optionfilter2]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=option<T>%2C+(T+->+bool)+->+option<T>&filter-crate=std ## Motivation When type-based search was first landed, it was directly [described as incomplete][a comment]. [a comment]: rust-lang#23289 (comment) Filling out the missing functionality is going to mean adding support for more of Rust's [type expression] syntax, such as references, raw pointers, function pointers, and closures. This PR adds function pointers and closures. [type expression]: https://doc.rust-lang.org/reference/types.html#type-expressions There's been demand for something "like Hoogle, but for Rust" expressed a few times [1](https://www.reddit.com/r/rust/comments/y8sbid/is_there_a_website_like_haskells_hoogle_for_rust/) [2](https://users.rust-lang.org/t/rust-equivalent-of-haskells-hoogle/102280) [3](https://internals.rust-lang.org/t/std-library-inclusion-policy/6852/2) [4](https://discord.com/channels/442252698964721669/448238009733742612/1109502307495858216). Some of them just don't realize what functionality already exists ([`Duration -> u64`](https://doc.rust-lang.org/nightly/std/?search=duration%20-%3E%20u64) already works), but a lot of them specifically want to search for higher-order functions like option combinators. ## Guide-level explanation (from the Rustdoc book) To search for a function that accepts a function as a parameter, like `Iterator::all`, wrap the nested signature in parenthesis, as in [`Iterator<T>, (T -> bool) -> bool`][iterator-all]. You can also search for a specific closure trait, such as `Iterator<T>, (FnMut(T) -> bool) -> bool`, but you need to know which one you want. [iterator-all]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=Iterator<T>%2C+(T+->+bool)+->+bool&filter-crate=std ## Reference-level description (also from the Rustdoc book) ### Primitives with Special Syntax <table> <thead> <tr> <th>Shorthand</th> <th>Explicit names</th> </tr> </thead> <tbody> <tr><td colspan="2">Before this PR</td></tr> <tr> <td><code>[]</code></td> <td><code>primitive:slice</code> and/or <code>primitive:array</code></td> </tr> <tr> <td><code>[T]</code></td> <td><code>primitive:slice<T></code> and/or <code>primitive:array<T></code></td> </tr> <tr> <td><code>!</code></td> <td><code>primitive:never</code></td> </tr> <tr> <td><code>()</code></td> <td><code>primitive:unit</code> and/or <code>primitive:tuple</code></td> </tr> <tr> <td><code>(T)</code></td> <td><code>T</code></td> </tr> <tr> <td><code>(T,)</code></td> <td><code>primitive:tuple<T></code></td> </tr> <tr><td colspan="2">After this PR</td></tr> <tr> <td><code>(T, U -> V, W)</code></td> <td><code>fn(T, U) -> (V, W)</code>, Fn, FnMut, and FnOnce</td> </tr> </tbody> </table> The `->` operator has lower precedence than comma. If it's not wrapped in brackets, it delimits the return value for the function being searched for. To search for functions that take functions as parameters, use parenthesis. ### Search query grammar ```ebnf ident = *(ALPHA / DIGIT / "_") path = ident *(DOUBLE-COLON ident) [BANG] slice-like = OPEN-SQUARE-BRACKET [ nonempty-arg-list ] CLOSE-SQUARE-BRACKET tuple-like = OPEN-PAREN [ nonempty-arg-list ] CLOSE-PAREN arg = [type-filter *WS COLON *WS] (path [generics] / slice-like / tuple-like) type-sep = COMMA/WS *(COMMA/WS) nonempty-arg-list = *(type-sep) arg *(type-sep arg) *(type-sep) [ return-args ] generic-arg-list = *(type-sep) arg [ EQUAL arg ] *(type-sep arg [ EQUAL arg ]) *(type-sep) normal-generics = OPEN-ANGLE-BRACKET [ generic-arg-list ] *(type-sep) CLOSE-ANGLE-BRACKET fn-like-generics = OPEN-PAREN [ nonempty-arg-list ] CLOSE-PAREN [ RETURN-ARROW arg ] generics = normal-generics / fn-like-generics return-args = RETURN-ARROW *(type-sep) nonempty-arg-list exact-search = [type-filter *WS COLON] [ RETURN-ARROW ] *WS QUOTE ident QUOTE [ generics ] type-search = [ nonempty-arg-list ] query = *WS (exact-search / type-search) *WS ; unchanged parts of the grammar, like the full list of type filters, are omitted ``` ## Future direction ### The remaining type expression grammar As described in rust-lang#118194, this is another step in the type expression grammar: BareFunction, and the function-like mode of TypePath, are now supported. * RawPointerType and ReferenceType actually are a priority. * ImplTraitType and TraitObjectType (and ImplTraitTypeOneBound and TraitObjectTypeOneBound) aren't as much of a priority, since they desugar pretty easily. ### Search subtyping and traits This is the other major factor that makes it less useful than it should be. * `iterator<result<t>> -> result<t>` doesn't find `Result::from_iter`. You have to search [`intoiterator<result<t>> -> result<t>`](https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=intoiterator%3Cresult%3Ct%3E%3E%20-%3E%20result%3Ct%3E&filter-crate=std). Nobody's going to search for IntoIterator unless they basically already know about it and don't need the search engine anyway. * Iterator combinators are usually structs that happen to implement Iterator, like `std::iter::Map`. To solve these cases, it needs to look at trait implementations, knowing that Iterator is a "subtype of" IntoIterator, and Map is a "subtype of" Iterator, so `iterator -> result` is a subtype of `intoiterator -> result` and `iterator<t>, (t -> u) -> iterator<u>` is a subtype of [`iterator<t>, (t -> u) -> map<t -> u>`](https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=iterator%3Ct%3E%2C%20(t%20-%3E%20u)%20-%3E%20map%3Ct%20-%3E%20u%3E&filter-crate=std).

…uillaumeGomez rustdoc-search: search types by higher-order functions This feature extends rustdoc with syntax and search index information for searching function pointers and closures (Higher-Order Functions, or HOF). Part of rust-lang/rust#60485 This PR adds two syntaxes: a high-level one for finding any kind of HOF, and a direct implementation of the parenthesized path syntax that Rust itself uses. ## Preview pages | Query | Results | |-------|---------| | [`option<T>, (fnonce (T) -> bool) -> option<T>`][optionfilter] | `Option::filter` | | [`option<T>, (T -> bool) -> option<T>`][optionfilter2] | `Option::filter` | Updated chapter of the book: https://notriddle.com/rustdoc-html-demo-9/search-hof/rustdoc/read-documentation/search.html [optionfilter]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=option<T>%2C+(fnonce+(T)+->+bool)+->+option<T>&filter-crate=std [optionfilter2]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=option<T>%2C+(T+->+bool)+->+option<T>&filter-crate=std ## Motivation When type-based search was first landed, it was directly [described as incomplete][a comment]. [a comment]: rust-lang/rust#23289 (comment) Filling out the missing functionality is going to mean adding support for more of Rust's [type expression] syntax, such as references, raw pointers, function pointers, and closures. This PR adds function pointers and closures. [type expression]: https://doc.rust-lang.org/reference/types.html#type-expressions There's been demand for something "like Hoogle, but for Rust" expressed a few times [1](https://www.reddit.com/r/rust/comments/y8sbid/is_there_a_website_like_haskells_hoogle_for_rust/) [2](https://users.rust-lang.org/t/rust-equivalent-of-haskells-hoogle/102280) [3](https://internals.rust-lang.org/t/std-library-inclusion-policy/6852/2) [4](https://discord.com/channels/442252698964721669/448238009733742612/1109502307495858216). Some of them just don't realize what functionality already exists ([`Duration -> u64`](https://doc.rust-lang.org/nightly/std/?search=duration%20-%3E%20u64) already works), but a lot of them specifically want to search for higher-order functions like option combinators. ## Guide-level explanation (from the Rustdoc book) To search for a function that accepts a function as a parameter, like `Iterator::all`, wrap the nested signature in parenthesis, as in [`Iterator<T>, (T -> bool) -> bool`][iterator-all]. You can also search for a specific closure trait, such as `Iterator<T>, (FnMut(T) -> bool) -> bool`, but you need to know which one you want. [iterator-all]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=Iterator<T>%2C+(T+->+bool)+->+bool&filter-crate=std ## Reference-level description (also from the Rustdoc book) ### Primitives with Special Syntax <table> <thead> <tr> <th>Shorthand</th> <th>Explicit names</th> </tr> </thead> <tbody> <tr><td colspan="2">Before this PR</td></tr> <tr> <td><code>[]</code></td> <td><code>primitive:slice</code> and/or <code>primitive:array</code></td> </tr> <tr> <td><code>[T]</code></td> <td><code>primitive:slice<T></code> and/or <code>primitive:array<T></code></td> </tr> <tr> <td><code>!</code></td> <td><code>primitive:never</code></td> </tr> <tr> <td><code>()</code></td> <td><code>primitive:unit</code> and/or <code>primitive:tuple</code></td> </tr> <tr> <td><code>(T)</code></td> <td><code>T</code></td> </tr> <tr> <td><code>(T,)</code></td> <td><code>primitive:tuple<T></code></td> </tr> <tr><td colspan="2">After this PR</td></tr> <tr> <td><code>(T, U -> V, W)</code></td> <td><code>fn(T, U) -> (V, W)</code>, Fn, FnMut, and FnOnce</td> </tr> </tbody> </table> The `->` operator has lower precedence than comma. If it's not wrapped in brackets, it delimits the return value for the function being searched for. To search for functions that take functions as parameters, use parenthesis. ### Search query grammar ```ebnf ident = *(ALPHA / DIGIT / "_") path = ident *(DOUBLE-COLON ident) [BANG] slice-like = OPEN-SQUARE-BRACKET [ nonempty-arg-list ] CLOSE-SQUARE-BRACKET tuple-like = OPEN-PAREN [ nonempty-arg-list ] CLOSE-PAREN arg = [type-filter *WS COLON *WS] (path [generics] / slice-like / tuple-like) type-sep = COMMA/WS *(COMMA/WS) nonempty-arg-list = *(type-sep) arg *(type-sep arg) *(type-sep) [ return-args ] generic-arg-list = *(type-sep) arg [ EQUAL arg ] *(type-sep arg [ EQUAL arg ]) *(type-sep) normal-generics = OPEN-ANGLE-BRACKET [ generic-arg-list ] *(type-sep) CLOSE-ANGLE-BRACKET fn-like-generics = OPEN-PAREN [ nonempty-arg-list ] CLOSE-PAREN [ RETURN-ARROW arg ] generics = normal-generics / fn-like-generics return-args = RETURN-ARROW *(type-sep) nonempty-arg-list exact-search = [type-filter *WS COLON] [ RETURN-ARROW ] *WS QUOTE ident QUOTE [ generics ] type-search = [ nonempty-arg-list ] query = *WS (exact-search / type-search) *WS ; unchanged parts of the grammar, like the full list of type filters, are omitted ``` ## Future direction ### The remaining type expression grammar As described in rust-lang/rust#118194, this is another step in the type expression grammar: BareFunction, and the function-like mode of TypePath, are now supported. * RawPointerType and ReferenceType actually are a priority. * ImplTraitType and TraitObjectType (and ImplTraitTypeOneBound and TraitObjectTypeOneBound) aren't as much of a priority, since they desugar pretty easily. ### Search subtyping and traits This is the other major factor that makes it less useful than it should be. * `iterator<result<t>> -> result<t>` doesn't find `Result::from_iter`. You have to search [`intoiterator<result<t>> -> result<t>`](https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=intoiterator%3Cresult%3Ct%3E%3E%20-%3E%20result%3Ct%3E&filter-crate=std). Nobody's going to search for IntoIterator unless they basically already know about it and don't need the search engine anyway. * Iterator combinators are usually structs that happen to implement Iterator, like `std::iter::Map`. To solve these cases, it needs to look at trait implementations, knowing that Iterator is a "subtype of" IntoIterator, and Map is a "subtype of" Iterator, so `iterator -> result` is a subtype of `intoiterator -> result` and `iterator<t>, (t -> u) -> iterator<u>` is a subtype of [`iterator<t>, (t -> u) -> map<t -> u>`](https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=iterator%3Ct%3E%2C%20(t%20-%3E%20u)%20-%3E%20map%3Ct%20-%3E%20u%3E&filter-crate=std).

rust-highfive assigned pcwalton Mar 11, 2015

alexcrichton assigned alexcrichton and unassigned pcwalton Mar 11, 2015

alexcrichton reviewed Mar 13, 2015
View reviewed changes

mihneadb reviewed Mar 13, 2015
View reviewed changes

alexcrichton reviewed Mar 13, 2015
View reviewed changes

bors merged commit 7b7b938 into rust-lang:master Mar 15, 2015

piatra mentioned this pull request May 6, 2015

Add note about type signature searching to docs #25146

Merged

mihneadb mentioned this pull request Jun 14, 2015

Find type of orphan methods for rustdoc search. #26293

Merged

mihneadb mentioned this pull request Jun 16, 2015

[Rustdoc] save generic type data to index + basic search #26356

Closed

notriddle mentioned this pull request Mar 1, 2023

rustdoc: Allow whitespace as path separator like double colon #108537

Merged

notriddle mentioned this pull request May 26, 2023

rustdoc: search for slices and arrays by type with [] #111958

Merged

notriddle mentioned this pull request Nov 23, 2023

rustdoc: search for tuples and unit by type with () #118194

Merged

notriddle mentioned this pull request Jan 7, 2024

rustdoc-search: search types by higher-order functions #119676

Merged

cjquines mentioned this pull request Mar 19, 2024

feat: add pullObject remeda/remeda#576

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rustdoc search by type #23289

Rustdoc search by type #23289

mihneadb commented Mar 11, 2015

rust-highfive commented Mar 11, 2015

mihneadb commented Mar 11, 2015

steveklabnik commented Mar 11, 2015

mihneadb commented Mar 11, 2015

jdm commented Mar 11, 2015

killercup commented Mar 11, 2015

mihneadb commented Mar 12, 2015

killercup commented Mar 12, 2015

mihneadb commented Mar 12, 2015

alexcrichton Mar 13, 2015

mihneadb commented Mar 13, 2015

mihneadb Mar 13, 2015

alexcrichton Mar 13, 2015

mihneadb Mar 13, 2015

jstasiak commented Mar 13, 2015

alexcrichton commented Mar 13, 2015

mihneadb commented Mar 13, 2015

mihneadb commented Mar 13, 2015

alexcrichton Mar 13, 2015

mihneadb Mar 13, 2015

steveklabnik commented Mar 14, 2015

bors commented Mar 14, 2015

bors commented Mar 14, 2015

bors commented Mar 15, 2015

Rustdoc search by type #23289

Rustdoc search by type #23289

Conversation

mihneadb commented Mar 11, 2015

rust-highfive commented Mar 11, 2015

mihneadb commented Mar 11, 2015

steveklabnik commented Mar 11, 2015

mihneadb commented Mar 11, 2015

jdm commented Mar 11, 2015

killercup commented Mar 11, 2015

mihneadb commented Mar 12, 2015

killercup commented Mar 12, 2015

mihneadb commented Mar 12, 2015

alexcrichton Mar 13, 2015

Choose a reason for hiding this comment

mihneadb commented Mar 13, 2015

mihneadb Mar 13, 2015

Choose a reason for hiding this comment

alexcrichton Mar 13, 2015

Choose a reason for hiding this comment

mihneadb Mar 13, 2015

Choose a reason for hiding this comment

jstasiak commented Mar 13, 2015

alexcrichton commented Mar 13, 2015

mihneadb commented Mar 13, 2015

mihneadb commented Mar 13, 2015

alexcrichton Mar 13, 2015

Choose a reason for hiding this comment

mihneadb Mar 13, 2015

Choose a reason for hiding this comment

steveklabnik commented Mar 14, 2015

bors commented Mar 14, 2015

bors commented Mar 14, 2015

bors commented Mar 15, 2015