Piping from less into ChromatTerm does not keep less open interactively

[aaronols]

Description

A command of the structure less file.log | ct correctly opens the desired file in less and colorizes it.
The less program is however not interactive, scrolls through the entire log file and then closes itself.
What do I have to do to keep the less program interactive when piping from less into ct?
I have looked over the man page for less but could not find a flag to force interactive mode or something similar.

Environment

• Terminal: zsh in tmux on wsltty
• OS: [Ubuntu 18.04.3 LTS @ WSL @ Win10 1803 build 17134.1039]
• ChromaTerm: [0.5.1]

Configuration

This happens with the default configuration

[hSaria]

I see you've already performed the initial investigation :)

Ideally, ct would set the tty of the standard input to that of the standard output, thus making ct invisible while supporting further piping. However, I'm not sure if that's possible. I'll do a bit more research on this.

I've thought of having ct spawn the process, like ct less file.log in order to have more control over the internal pipes, but that wouldn't help either as the stdout of the spawned program would still have to be redirected, never mind the additional overhead and complexity of this approach.

Anyway, let me fix #75 first as it's more straight forward and then I'll give this the attention it requires.

Thanks for opening the issues. I greatly appreciate it.

[hSaria]

I've been working on this and while I do have POC code that fundamentally works, I don't think it's acceptable due to its very risky approach. I've looked at the following options but each has its own drawbacks:

1. os.fork and os.openpty, then use subprocess to start up a process that has its stdout set to the tty from os.openpty. The availability on that one is limited to "some flavors of Unix" which I really don't like the sounds of. It could all work well up until the point where it doesn't with my hands tied.

2. Using a similar approach to the one above, but with the help of the pty module to replace the os calls. I looked at the code for that module and, unfortunately, it utilizes some non-deterministic methods or plain workarounds when support for the native os calls is not present. At that point, I'm better off with my first option.

3. Using the pty.spawn call to minimize the amount of custom code. This is even worse than option 2 as there are known bugs around its behaviour, namely the terminal's state not being forwarded to the spawned program. It also utilizes pretty much all of the functions in the library to achieve its goal, so I'd still be stuck with the same workarounds.

I've also tried combinations of the above, but the code starts looking sketchy.

Essentially, I'm trying to

1. create a file descriptor that is considered a tty (i.e. os.isatty(some_fd)),
2. spawn a new process (fork or otherwise) where a program (e.g. less) has its stdout set to the tty file descriptor,
3. read from the file descriptor that the spawned process is writing/outputting to, then
4. finally, process the data that was read and print it to stdout.

The good news is that I can already do steps 2, 3, and 4 cleanly (as intended). I'm just trying to find the best way – or as close to it as possible – to creating a tty file descriptor that would allow me to make ChromaTerm invisible.

I look for a bit more.

[hSaria]

I ran into /dev/ptmx though that will only get me one half of the tty (master). Unfortunately, I can't use this method because getting the other side isn't possible in Python (can't get the slave with ptsname).

It would appear the os.openpty is actually just using the posix standard. During compilation of Python, it's including the header file where openpty lives. I'm more than happy with that as support for posix is somewhat safe to rely on.

I'll work on making this work. You'll be able to do things like ct --run less file.log (any command prefixed with ct --run).

[hSaria]

TLDR: got it all working; just reviewing the code and writing up tests. Should be done soon.

I've put some text in the epilog of ct -h that explains when to use --run. Normally, you should just pipe.

$ ct -h
usage: ct [-h] [--config FILE] [--reload] [--rgb] [--run ...]

optional arguments:
  -h, --help     show this help message and exit
  --config FILE  location of config file (default: $HOME/.chromaterm.yml)
  --reload       Reload the config of all CT instances
  --rgb          Use RGB colors (default: detect support, fallback to
                 xterm-256)
  --run ...      run a program with anything after it used as arguments

ChromaTerm reads from standard input; just pipe data to it. As this exposes
the existance of a pipe to the piping process, ChromaTerm can --run your
program in order to hide the pipe. This is normally only needed for programs
that want to be on a controlling terminal, like `less`.

I've written a test script for verifying if the stdin and stdout are tty's from the program's perspective.

Here's what normal program looks like when called from within a shell (both are tty's or True):

$ ./test_tty.py 
stdin=True, stdout=True

When piping, a program will have the ability to detect that stdin and/or stdout are not tty's.

$ ./test_tty.py | cat
stdin=True, stdout=False
$ echo | ./test_tty.py | cat
stdin=False, stdout=False

When using --run, the stdout of a program uses a tty created by ct; stdin remains untouched.

$ ct --run ./test_tty.py 
stdin=True, stdout=True

If you try to pipe stdin and/or stdout when using ct --run, the stdin behavior is no different to what currently happens. It's only stdout which is modified.

$ echo | ct --run ./test_tty.py 
stdin=False, stdout=True
$ echo | ct --run ./test_tty.py | cat
stdin=False, stdout=True

[hSaria]

Completed in a76a348...a52fe3a. v0.5.2 will be published soon.

BTW, I removed the need for --run, so you can just prepend your command with ct to color it.

[aaronols]

Thank you very much for your time in fixing this. Can confirm that less run by chromaterm is now interactive 👍

One little thing to improve: The positional arguments are not listed in the usage explanation line in the help section:

-usage: ct [-h] [--config FILE] [--reload] [--rgb] ...
+usage: ct [-h] [--config FILE] [--reload] [--rgb] [program] ...

positional arguments:
  program        run a program with anything after it used as argument

[hSaria]

Thanks for the suggestion. I’ll add this in for the next version. I’m using the default help formatter of ArgParse; the help output is auto-generated. The ... is how ArgParse represents a capture-remaining argument. I can, however, add a dedicated argument for the program to make the help output clearer.

…

On 21 Oct 2019, at 8:17 am, aaronols ***@***.***> wrote:  Thank you very much for your time in fixing this. Can confirm that less run by chromaterm is now interactive 👍 One little thing to improve: The positional arguments are not listed in the usage explanation line in the help section: -usage: ct [-h] [--config FILE] [--reload] [--rgb] ... +usage: ct [-h] [--config FILE] [--reload] [--rgb] [program] ... positional arguments: program run a program with anything after it used as argument — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.

[hSaria]

Clarified the help output in 9511bf6. Now it looks like this:

usage: ct [-h] [--config FILE] [--reload] [--rgb] [program ...]

positional arguments:
  program ...    run a program with anything after it used as arguments

...

Once more, thank you for the feedback.