Inconsistent highlighting of single letters

[Enf0]

Easier to explain with a screenshot:

It seems that the highlight rules for RIP, EIGRP, BGP and OSPF matches the single letters in the Cisco CLI prompt, even though it shouldn't match if these letters are part of a word, i.e. not standalone?

Is this a bug, or is this supposed to happen given the regex below?

- description: BGP Part 2
  regex: \b(%(BGP|BGP_SESSION)\-\d\-\w+|bgp|BGP|B|IGP|incomplete|\d{2,7}\/nolabel\(\w+\)|RR\-client|Originator|cluster\-id|Cluster\-id|Cluster|Route\-Reflector)\b
  color: f#AF7AC5

- description: OSPFv2 and OSPFv3
  regex: \b(OSPF_VL\d{1,2}|OSPF_SL\d{1,2}|VL\d{1,2}|SL\d{1,2}|Type\-\d|ospf|OSPF|O|IA|E[12]|N[12]|P2P|P2MP|BDR|DR|ABR|ASBR|LOOP|DROTHER|POINT_TO_POINT|POINT_TO_MULTIPOINT|BROADCAST|NON_BROADCAST|LOOPBACK|SHAM_LINK|3101|1587|transit|Transit|nssa|NSSA|stub|Stub|Superbackbone|OSPFv3_VL\d{1,2}|OSPFv3\-\d{1,5}\-IPv6|ospfv3|OSPFv3|OI|OE[12]|ON[12]|V6\-Bit|E\-Bit|R\-bit|DC\-Bit|opaque|DROTH|%OSPF(V3)?\-\d\-\w+)\b
  color: f#ffa500

- description: EIGRP
  regex: \b(EIGRP\-IPv6|EIGRP\-IPv4|eigrp|EIGRP|EX|D|K[13]=1|K[245]=0|Internal|External|%DUAL\-\d\-\w+)\b
  color: f#008080

- description: RIP
  regex: \b((rip|RIP|R))\b
  color: f#ff0000

[hSaria]

It's a weird behavior with using single-letter matching and how ChromaTerm processes the text. (A bit lengthy; sorry.)

As you type on your keyboard (stdin) and the input is processed and sent back to your screen (i.e. `stdout), ChromaTerm will take that and attempt to highlight each part of the text individually, split by separators. A common example of a separator is new line. Basically, it splits the input into chunks that will *independently be matched against the rules.

Now, what if the data doesn't end in a separator? Well, ChromaTerm will wait a very small amount (0.05 ms or 500 µsec) to make sure it won't be receiving any more data, and – if it doesn't – it will processes it as its own chunk.

While typing, each character is basically its own chunk (unless you're typing faster than 500 µsec, in which case you are a literal keyboard god). As you type and the remote server sends back output, here's what happens:

1. You press a key, like R
2. The remote end (e.g. SSH) processes it and sends it back to stdout
3. The stdout of the remote end is forwarded (piped) to ChromaTerm
4. ChromaTerm notices that the text it received could not be separated and so there might be more data coming in; it waits a small amount of time (500 µsec) for more data
5. The small amount of time expires and ChromaTerm now processes the text R
6. Based on the rules, something like \bR\b is matching R

This complicated work is needed because ChromaTerm must work with interactive applications. Otherwise, we could just line-buffer everything and call it a day.

I may be able to apply some logic to prevent this (i.e. processing unseparated text), but I'm still thinking whether or not they're realistically achievable without any work on the user's side.

[hSaria]

BTW, here's where this logic is present:

ChromaTerm/chromaterm/__init__.py

Lines 211 to 222 in 8a9774d

	for split in splits[:-1]: # Process all splits except for the last
	print(highlight(config, split[0]) + split[1], end='')
	# Indicated more data to possibly come and read_fd confirmed it
	if more and read_ready(config.get('read_fd'), WAIT_FOR_SPLIT):
	# Return last split as the left-over data
	return splits[-1][0] + splits[-1][1]
	# No more data; print last split and flush as it doesn't have a new line
	print(highlight(config, splits[-1][0]) + splits[-1][1], end='', flush=True)
	return '' # All of the buffer was processed; return an empty buffer

The simplest solution would be not to highlight any text not ending a separator and just print it as is. This will work fine as just about everything will contain a separator. The only exception to this is if the piping program's last line (when it closes) doesn't have a separator, like so:

# "there" would normally be highlighted but since it doesn't end in a separator, it won't 
echo -n "Hello there, World" | ct

# echo implicitly adds a new line at the end, which is a separator, so "there" is highlighted.
echo "Hello there, World" | ct

Basically, this

diff --git a/chromaterm/__init__.py b/chromaterm/__init__.py
index 226b0d3..a0858dc 100644
--- a/chromaterm/__init__.py
+++ b/chromaterm/__init__.py
@@ -219,3 +219,3 @@ def process_buffer(config, buffer, more):
     # No more data; print last split and flush as it doesn't have a new line
-    print(highlight(config, splits[-1][0]) + splits[-1][1], end='', flush=True)
+    print(splits[-1][0] + splits[-1][1], end='', flush=True)

[Enf0]

Thank you for the very detailed response! Now I understand why this happens. I realize that trying to fix this might not be straight-forward, either way it is a minor issue for me.

Thanks again.

[hSaria]

I'm gonna reopen it and mark it as an enhancement (nearly ready).

[hSaria]

Fixed and will be published in v0.5.8. Thanks a lot for your recent feedback/submissions; I greatly appreciate them. This one, for example, also helped me pick a scientifically better value for the delay, instead of the arbitrary 500 µsec.

[Enf0]

Brilliant! You rock. And it's my pleasure, as soon as I discovered ChromaTerm and realized I could use a proper terminal emulator (I use Windows Terminal and WSL now) instead of SecureCRT, I was so happy.
I've already converted one of my co-workers, and several people have asked me how I get my terminal so pretty ;)