Long filename contains unicode characters with by-file truncates error

[lingsamuel]

Describe the bug
Long filename contains unicode characters with by-file truncated incorrectly.

To Reproduce

touch "ббббббббббббббббббббббббббббббббб.md"
touch "ѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬѬ.md"
touch "中文中文中文中文中文中文中文中文中文中文中文中文中文中文中文中文.md"
scc --by-file

Output:

Expected behavior
�� shouldn't appear, indent should works properly.

Desktop (please complete the following information):

• OS: Linux 5.9.9-arch1-1 (Arch Linux)
• scc version 2.13.0, make from source

[lingsamuel]

б https://www.compart.com/en/unicode/U+0431
Ѭ https://www.compart.com/en/unicode/U+046C
中 https://www.compart.com/en/unicode/U+4E2D

[boyter]

You would think I would be aware of this considering how much time I have spent looking over unicode recently.

Its down to it using bytes and not characters. Its especially annoying because it trims from the front, which isn't how Go lets you walk runes.... but not impossible to rectify. I can see the size never matching up though depending on the characters though, but we can fix the current issues for sure.

[boyter]

Done. Note there isn't much I can do about 中 which for some reason does not display in mono-spaced font... tokei has the same issue. However at least its not cut off anymore when printing.

If its urgent, a build from master should resolve this for you :)

[dbaggerman]

Note there isn't much I can do about 中 which for some reason does not display in mono-spaced font

You could have a play around with https://godoc.org/golang.org/x/text/width to calculate the display width.

[boyter]

I actually found this just then http://github.com/mattn/go-runewidth which might be able to solve this.

[boyter]

Added the above in... and its all there.. however it appears sprintf has issues with this as well... but moved over to some custom parsing. Anyway merged in to master.

[lingsamuel]

Thanks!

[boyter]

@lingsamuel No problem at all.