Panic on trying to highlight new file (even when empty)

[yaksher]

rust-analyzer version: rust-analyzer version: 0.3.1730-standalone (416e9c8 2023-11-12)

rustc version: rustc 1.73.0 (cc66ad468 2023-10-03)

relevant settings: No settings that I can think of being relevant

Adding a new file to a project causes rust-analyzer to fail to analyze that file and print a panic to output. This happens both when the file is empty and when rerunning with code in the file.

Panic context:
> 
version: 0.3.1730-standalone (416e9c856 2023-11-12)
request: textDocument/documentHighlight DocumentHighlightParams {
    text_document_position_params: TextDocumentPositionParams {
        text_document: TextDocumentIdentifier {
            uri: Url {
                scheme: "file",
                cannot_be_a_base: false,
                username: "",
                password: None,
                host: None,
                port: None,
                path: "/Users/yaksher/rustbucket/rustbucket/src/hashbucket/hashbucket.rs",
                query: None,
                fragment: None,
            },
        },
        position: Position {
            line: 0,
            character: 7,
        },
    },
    work_done_progress_params: WorkDoneProgressParams {
        work_done_token: None,
    },
    partial_result_params: PartialResultParams {
        partial_result_token: None,
    },
}

thread 'Worker' panicked at /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/cursor.rs:751:9:
Bad offset: range 0..1 offset 7
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: rowan::cursor::SyntaxNode::token_at_offset
   3: ide::highlight_related::highlight_related
   4: std::panicking::try
   5: rust_analyzer::handlers::request::handle_document_highlight
   6: core::ops::function::FnOnce::call_once{{vtable.shim}}
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
[Error - 3:04:10 PM] Request textDocument/documentHighlight failed.
  Message: request handler panicked: Bad offset: range 0..1 offset 7
  Code: -32603 

[alibektas]

So to clarify : Is the newly created file called hashbucket.rs? And btw your linking of this issue to 15865 may not be the right call for now as we have so little evidence that points a relation between them

[yaksher]

Yes, hashbucket is the file newly added. And sorry about that, I was just linking it because it technically matched the provided description.

[alibektas]

And sorry about that, I was just linking it because it technically matched the provided description.

No worries at all, just saying.

and content-wise you say that the file is empty, interesting. the reason I am asking is that the error stems from our parser not being able to find a token at a given offset and this is strange. Maybe you could share only the first couple of lines of hashbucket.rs because you said rust-analyzer also fails when the file is populated.

[yaksher]

I’ve deleted the file in the attempts to get it to work but it failed with stuff like (entire contents of the file, plus or minus a couple empty lines)

struct HashBucket {}

impl HashBucket {}

and variations.

[yaksher]

Some additional context is that it would pop up about something like detecting the Cargo.toml file up stream in ~/rustbucket/rustbucket/and asking if I wanted to add the file to the project with options "yes", "no", "don't show this again"—when I clicked "yes" it would print that to VS Code output; the prompt would then pop up again if I typed a character. I eventually clicked "don't show this again," so it would be annoying to reproduce the output.

[Veykril]

So there are two possible things that are going on here, either vscode was sending a highlight request with an out of bounds offset, or we were trying to retry an old highlight request after a file got modified which means the offsets are wrong, The first one is a VSCode bug, the second case ours. Given the description of the issue, it seems to be the forme rthough. (nevertheless we should catch this in some way)

[alibektas]

On it. But I thought it points to an error of the latter kind which must be handled in some other way. So I am just going to return None in highlight_related if that's ok?

[Veykril]

We need to handle this in the protocol layer instead such that this generally fixes things. That is we need to attach a document version to requests and request-retry tasks and verify that they are still valid when we retry them.

If invalid, we either need to fail the request, or better try to fix up any offsets inside it which is a lot harder and has to be done on a per request basis. So the former should suffice for the time being.

[yaksher]

Oh, something I realized that's definitely worth noting is that the issue stopped happening when I used mod hashbucket (well, mod hashbucket {pub mod hashbucket;}) in main.rs to actually add the file to the module-level dependency tree of the project instead of just having it hanging out in there with nothing referring to.