Add support for XTGETTCAP

[kchibisov]

How it works is available at https://invisible-island.net/xterm/ctlseqs/ctlseqs.html .

This is used to query for terminfo features and not relying on the actual files. It's supported by notcursors, most modern terms, etc. It could be sort of good when you ssh into system without TERM info you have and it could help terminals not having their terminfo spread to work with at least modern toolkits.

[chrisduerr]

Not a fan of this at all. Why do you want this?

[kchibisov]

mostly because it started to get used in toolkits.

I'd at least like to look into what it is a bit.

[chrisduerr]

mostly because it started to get used in toolkits.

Which one?

[kchibisov]

@chrisduerr I think I mentioned? it's notcurses, some recent go toolkits, etc. It's also present in basically any active terminal.

[roland-5]

From my experience, the currently released Neovim 0.10 uses XTGETTCAP as the escape sequence to detect OSC 52 support automatically and thus clipboard function within itself. This is very useful for me when I want to copy code from one remote machine and quickly paste it into another to check something.

[dcolascione]

Why wouldn't we want all terminal emulators to support this mechanism? TERM has serious long-term problems.

[chrisduerr]

Why wouldn't we want all terminal emulators to support this mechanism? TERM has serious long-term problems.

Because this also has serious problems, which is why it hasn't really been used by anything for like a century. But people got fed up with trying to make something new work so instead they're trying to switch from one set of problems to another. At the end of the day this just creates fragmentation with no benefits.

[dcolascione]

What are the serious problems?

[j4james]

99% of the features identified in a modern terminfo file are just standard ANSI or DEC sequences, or commonly supported Xterm extensions, that every modern terminal implements. This is why so many terminals can get away with just setting their TERM to xterm-256colors. Inventing a proprietary escape sequence for apps to look up a bunch of ANSI-standard escape sequences is patently ridiculous.

And if you're going to say that you're not using XTGETTCAP to query all those standard escape sequences, then why on earth would you expect terminals to report them? It's like asking someone their favorite color, but demanding that they provide the answer in the form of a 50,000 word novel, which you don't intend to read.

If there is a feature that isn't universally supported (like say OSC 52), there are better ways to report that than XTGETTCAP. You could propose a new DA1 feature, or a mode that can be queried with DECRQM, or even just extend the OSC 52 sequence itself with a feature reporting option. For example, if the first parameter is ?, then the terminal could report back the subset of OSC 52 that it can handle (not everyone supports paste, and even those that support copying don't all support the same clipboard targets).

[j4james]

Btw, notcurses is another example of unnecessary XTGETTCAP usage. The only capabilities it's querying are RGB and hpa. The former is used as a proxy for 24-bit SGR color support, which can be more accurately determined with a standard DECRQSS query. And hpa could be determined with a simple DSR-CPR test (i.e. attempt to move the cursor with HPA, and check whether it has actually moved with DSR-CPR).

Expecting terminals to implement all of the cruft associated with XTGETTCAP just so you can query two features that can already be determined via existing standards is absurd.

[kchibisov]

OSC 52 is a part of terminfo though already and was that for a while, I have no idea why neovim decided to not use terminfo as a fallback for it, sounds like just a push for XTGETTCAP just because they want to. And new flags are added to the terminfo, this XTGETTCAP is basically report a terminfo value, so those two are in sync.

The only real place where it kind of matters, is when you ssh into remote without your terminfo installed or with some dated version of it.

I'd state again that XTGETTCAP is just a terminfo reporting, so they use the same base file as a source of truth, it's not like one format is better, because they use the same database under the hood, it's just could be done via ssh and the other requires your hosts to be up to date/synced to some extent.

[j4james]

OSC 52 is a part of terminfo though

But it's such a convoluted definition considering what little information it provides. For Xterm it's defined as Ms=\E]52;%p1%s;%p2%s\007, which suggests other terminals could use a completely different escape sequence. Except that they'd also have to use p1 and p2 parameters that are exactly compatible with Xterm, which isn't even true of many of the existing OSC 52 implementations.

And in the case of nvim, they basically treat it like a boolean capability - if the definition starts with \e]52, it's just assumed to be compatible with Xterm's implementation. So what was the point of that whole parameterized string definition?

I'd state again that XTGETTCAP is just a terminfo reporting.

And that's the root of the problem, because terminfo was a great idea maybe 40 years ago, when you were trying to write an app that could work on many different hardware terminals with wildly different escape sequences. In this day and age, assuming you're writing a modern application targeting modern software terminals, terminfo causes more problems than it solves.

That said, if you (Alacritty) actually want to support XTGETTCAP, you're welcome to do so. I was just under the impression that you were against the idea, and I was attempting to explain to the person above why it wasn't worth the effort. But if you disagree, feel free to disregard everything I've said.

[chrisduerr]

That said, if you (Alacritty) actually want to support XTGETTCAP, you're welcome to do so. I was just under the impression that you were against the idea, and I was attempting to explain to the person above why it wasn't worth the effort. But if you disagree, feel free to disregard everything I've said.

There's also issues beyond what you mentioned, like the question of which terminfo to report and the fact that you need to query non-standard stuff like Smulx separately since processing is interrupted as soon as any invalid name is found in the requested list.

[kchibisov]

That said, if you (Alacritty) actually want to support XTGETTCAP, you're welcome to do so. I was just under the impression that you were against the idea, and I was attempting to explain to the person above why it wasn't worth the effort. But if you disagree, feel free to disregard everything I've said.

I personally don't mind if certain apps really force the rest by starting doing so, but I just don't like it myself either.

[kchibisov]

This is very useful for me when I want to copy code from one remote machine and quickly paste it into another to check something.

can you force neovim to enable it? Sounds like there should be a way to do so.

[dcolascione]

The mechanism still seems pretty useful to me. With XTGETTCAP, programs request only the capabilities they need, so why would it hurt anything that information on , say, ANSI capabilities would be harmful, even if most terminals will report the same strings for those capabilities? IMHO, there's a lot of value in having a single converged capability detection mechanism that works for both old, boring capabilities and new, exciting ones. Maybe it'd be simpler to support only Boolean flags instead of parameterized capabilities, but IMHO not simpler enough that it's worth making an alternative feature detection system. I plan to add support for XTGETTCAP to Emacs term, FWIW. I want to solve the ssh problem once and for all.

[kchibisov]

boring capabilities and new, exciting ones

Do you understand that terminfo will have new exciting ones as well, since they have the same source and one is based on another? Unless you're talking about a new protocol, nothing will change, except ssh miss-matches, as said.

[kchibisov]

I plan to add support for XTGETTCAP to Emacs term, FWIW. I want to solve the ssh problem once and for all.

Solved with one scp | tic command or running systems that are not 5-7 years old.

[chrisduerr]

IMHO, there's a lot of value in having a single converged capability detection mechanism that works for both old, boring capabilities and new, exciting ones.

Well this escape isn't that, so I don't see the point.

[Samasaur1]

This is very useful for me when I want to copy code from one remote machine and quickly paste it into another to check something.

can you force neovim to enable it? Sounds like there should be a way to do so.

Yes, you can:

vim.g.clipboard = {
  name = 'OSC 52',
  copy = {
    ['+'] = require('vim.ui.clipboard.osc52').copy('+'),
    ['*'] = require('vim.ui.clipboard.osc52').copy('*'),
  },
  paste = {
    ['+'] = require('vim.ui.clipboard.osc52').paste('+'),
    ['*'] = require('vim.ui.clipboard.osc52').paste('*'),
  },
}

(see :help clipboard-osc52)