Instructions to get emojis to work (on Windows)

[toburger]

I am using Emojis for a project and everything worked perfectly fine on my machine.
After distributing the self executable to the customer he reported me that the emojis didn't get printed out and instead two question marks (??) are displayed.
This was quite surprising because in the documentation there exists no section of when emojis get printed and when not.

So this is an effort of collecting solutions on how to get emojis to work on Windows.

• Set the OutputEncoding manually:
  This is an easy fix to do, just add System.Console.OutputEncoding = System.Text.Encoding.UTF8 as the first line of the main function.
  Altough this is a quick fix I don't think it is the best solution because basically you override the Encoding settings of the parent console.

• Set the Encoding of the PowerShell (Core) console on a per session basis:
  You can set the input and output encoding of the console by executing the following line
  [console]::InputEncoding = [console]::OutputEncoding = [System.Text.Encoding]::UTF8
  This commaand can be put in the profile file so the command gets executed automatically.
  There exists also an issue to set UTF-8 as the default encoding on PowerShell Core: Make console windows fully UTF-8 by default on Windows, in line with the behavior on Unix-like platforms - character encoding, code page PowerShell/PowerShell#7233

• Set the UTF-8 worldwide language support:

  • Execute intl.cpl
  • Go to [Administrative], then [Change system locale...]
    
  • Set the checkbox
    

In fact this was the setting I had set on my development machine which caused to render the emojis correctly.

I hope this helps troubleshooting the issues I had with the output encoding (and emojis) and maybe some explanation could be added to the documentation.

[patriksvensson]

@toburger Thanks for opening this! Will update the documentation with this when I have some spare time.

[HolisticDeveloper]

I found that this affected the display of the line graphics as well, and not just the emojis. For example, having looked at the Progress sample, I tried implementing my own, but I was confused when the line styles and spinners did not match up with the ones in the animations. Setting the output encoding to utf-8 and running in Windows Terminal improved things.

[fredjeronimo]

It's worth pointing out that if you set the Console's output encoding, you should do so as early as possible or you might get artifacts with the terminal's width where Spectre.Console can fallback to a 80-character default.

Using Spectre.Console v0.37.0 for these use cases in a .NET Core 3.1 console application running in a Windows 10 system.

For example, the following is fine assuming that it's done at the very beginning of a console application:

Console.WindowWidth = 110;
Console.BufferWidth = 110;
Console.OutputEncoding = Encoding.UTF8;
Console.WriteLine($"Width: {AnsiConsole.Width}");   // 110

However, if instead you access any functionality in AnsiConsole beforehand (even accessing the width itself), you will get something unexpected:

Console.WindowWidth = 110;
Console.BufferWidth = 110;
Console.WriteLine($"Width (before OutputEncoding): {AnsiConsole.Width}");  // 110
Console.OutputEncoding = Encoding.UTF8;
Console.WriteLine($"Width (after OutputEncoding): {AnsiConsole.Width}");   // 80 (Constants.DefaultTerminalWidth)

This is because the standard .NET Console will flush and reset the internal Output writer - if Out is not being redirected - when the output encoding changes which leads to a completely new instance of the inner TextWriter being created, which in turn leads to the AnsiBackend falling back to the default width value (80 - Constants.DefaultTerminalWidth) because the TextWriterExtensions.IsStandardOut() check no longer matches for the lazily created inner _console instance in AnsiConsole.

public static Encoding OutputEncoding
{
    // [...]
    set
    {
         // [...]
         if (Volatile.Read(ref s_out) != null && !s_isOutTextWriterRedirected)
         {
             s_out.Flush();
             Volatile.Write(ref s_out, null);
         }
        // [...]
    }
}
// [...]
public static TextWriter Out => EnsureInitialized(ref s_out, () => CreateOutputWriter(OpenStandardOutput()));
// [...]

Another way to trigger this behaviour is to actually create a standard output redirection by setting a new Out in the standard Console:

Console.WindowWidth = 110;
Console.BufferWidth = 110;
Console.WriteLine($"Width (before SetOut): {AnsiConsole.Width}");  // 110
var originalOut = Console.Out;
Console.SetOut(new StringWriter());
var newWidth = AnsiConsole.Width;
Console.SetOut(originalOut);
Console.WriteLine($"Width (after SetOut): {newWidth}");            // 80 (Constants.DefaultTerminalWidth)

If for some reason, you need to change the console's output encoding in the middle of your application, one alternative is to explicitly re-create the AnsiConsole and use it from then on. Sadly, AnsiConsole.Console does not have a setter at present, so you are stuck with the one that is originally created on first usage for its static methods.

Console.WindowWidth = 110;
Console.BufferWidth = 110;
var oldAnsiConsole = AnsiConsole.Create(new AnsiConsoleSettings
{
    Ansi = AnsiSupport.Detect,
    ColorSystem = ColorSystemSupport.Detect,
    Out = Console.Out,
});
Console.OutputEncoding = Encoding.UTF8;
var newAnsiConsole = AnsiConsole.Create(new AnsiConsoleSettings
{
    Ansi = AnsiSupport.Detect,
    ColorSystem = ColorSystemSupport.Detect,
    Out = Console.Out,
});
Console.WriteLine($"Old AnsiConsole: {oldAnsiConsole.Width}");   // 80 (Constants.DefaultTerminalWidth)
Console.WriteLine($"New AnsiConsole: {newAnsiConsole.Width}");   // 110

[andtii]

@toburger @patriksvensson I cant get this to work on the default visual studio debug cmd window. Seems like the unicode does not work as expected, here i use the "Fira Code" font on all terminals but it only works on Windows Terminal. How do you do when you develop Spectre.Console @patriksvensson

[olabacker]

I also never got it working in vs debug window! Please share if you know how :)

[patriksvensson]

Two things determine whether or not Unicode characters can be shown:

1. The terminal encoding needs to be a Unicode encoding such as UTF-8.
2. The font needs to contain Unicode characters.

If the encoding isn't Unicode, writing Unicode characters to the terminal will show as garbage regardless of the font you're using.

To test this out in CMD.exe, try setting the code page to 65001 (UTF-8) using the chcp command or change it using System.Console.OutputEncoding as explained above.

That said, I don't think you should set the encoding explicitly as part of your application. The fallback mechanism to ASCII exists for a reason and will give the consumer of your application a much better experience.

To answer what I use when I develop Spectre.Console: I use Windows Terminal if I need to test Unicode stuff and cmd.exe for non-Unicode stuff. On macOS, I use iTerm2 and Alacrity.

I have no idea of how to get VS output window to behave correctly.

(Sorry for potential grammar issues and typos. Writing this on my phone)

[panmona]

I face this issue currently (with the new Windows Terminal app).
You (@patriksvensson) said:

I don't think you should set the encoding explicitly as part of your application.

I'm not sure though what else we can/should do to fix this or whether this is an issue of Spectre.Console. If so I'm interested what is going wrong there.

[patriksvensson]

Documentation added in #744

[CEbbinghaus]

I would like to add to this that the fallbacks are sometimes inadequate and that it would be really nice to allow for custom fallback bindings. In this case my cli does some reporting via ✅ & ❌ emojis. but while the rest of Spectre falls back to ASCII gracefully (The spinner still works perfectly), Those emoji's in particular are still rendered as ?

I know we can rebind emoji and I am tempted to just rebind those 2 to O & X with an if statement that triggers if the encoding isn't utf8, It would be much nicer to be able to just provide a mapping of emoji to Ascii that we can define ourslves. Willing to contribute to make such a feature happen