Unable to show Chinese Characters

[aDioS786]

My Issue/Question: I am trying to use ImGui::Selectable to dislplay English/Non-english names. But it is not letting me use u8 or wchar*. If i use char* then only english names are shown, the chinese names are Junk or shows as ???

So would like to know, how I can achieve this, i.e. it shows both characters perfectly fine. If i try to store non-english names as char* the most gives me junk values, whereas wchar* works fine. So not sure how I can achieve this. Thanks

[PathogenDavid]

@aDioS786 Without the details requested by the issue template we can only guess what your issue is. In particular a code snippet and the compiler you're using would be helpful.

But it is not letting me use u8

It sounds like you might be using C++20, which broke compatibility for UTF-8 strings. The exact fix depends on the compiler you're using.

[aDioS786]

@aDioS786 Without the details requested by the issue template we can only guess what your issue is. In particular a code snippet and the compiler you're using would be helpful.

But it is not letting me use u8

It sounds like you might be using C++20, which broke compatibility for UTF-8 strings. The exact fix depends on the compiler you're using.

Thanks for responding, Yes I'm using "ISO C++20 Standard (/std:c++20)" and VS 2022. I tried several ways to resolve but I couldn't succeed.

[ChivenZhang]

You need to add chinese glyph font into imgui.

io.Fonts->AddFontFromFileTTF("chinese font.ttf", font_size, nullptr, io.Fonts->GetGlyphRangesChineseFull());
io.Fonts->Build();


[ocornut]

Thanks for responding, Yes I'm using "ISO C++20 Standard (/std:c++20)" and VS 2022. I tried several ways to resolve but I couldn't succeed.

You can cast to const char*:

(const char*)u8"你好"

Likely you may want to wrap this in a macro:

#define _S(_LITERAL)    (const char*)u8##_LITERAL

ImGui::Text(_S("你好"));
ImGui::DebugTextEncoding(_S("你好"));

You can call DebugTextEncoding() with your string to confirm their encoding/contents.

Another solution may be to configure the compiler to use UTF-8 by default (not entirely sure this is possible, but see https://stackoverflow.com/questions/41335199/how-to-config-visual-studio-to-use-utf-8-as-the-default-encoding-for-all-project)

[PathogenDavid]

@aDioS786 Sorry I forgot to respond.

As an alternative to the _S workaround Omar posted, you can disable the breaking change with /Zc:char8_t-.

Another solution may be to configure the compiler to use UTF-8 by default (not entirely sure this is possible, but see https://stackoverflow.com/questions/41335199/how-to-config-visual-studio-to-use-utf-8-as-the-default-encoding-for-all-project)

You also need to enable /utf-8. Without it, the compiler will attempt to guess for each and every file and in the case of UTF-8 without a BOM it's always wrong.

[aDioS786]

Thanks both @ocornut and @PathogenDavid I will give both solutions a go and see if they work. I did tried to save files as UTF-8 @PathogenDavid but that didn't help. I'll try again around solutions. And will update. Thanks

[aDioS786]

Thanks for responding, Yes I'm using "ISO C++20 Standard (/std:c++20)" and VS 2022. I tried several ways to resolve but I couldn't succeed.

You can cast to const char*:

(const char*)u8"你好"

Likely you may want to wrap this in a macro:

#define _S(_LITERAL)    (const char*)u8##_LITERAL

ImGui::Text(_S("你好"));
ImGui::DebugTextEncoding(_S("你好"));

You can call DebugTextEncoding() with your string to confirm their encoding/contents.

Another solution may be to configure the compiler to use UTF-8 by default (not entirely sure this is possible, but see https://stackoverflow.com/questions/41335199/how-to-config-visual-studio-to-use-utf-8-as-the-default-encoding-for-all-project)

I've tried it and still unable to get it to work.

ImFont* font = io.Fonts->AddFontDefault();
DefaultFontRegular = io.Fonts->AddFontFromFileTTF("Fonts\\seguibl.ttf", 20, nullptr, io.Fonts->GetGlyphRangesChineseSimplifiedCommon());

//I have tried above line of code by passing fontConfig having merge field set to true as well, and have also tried adding GetGlyphRangesDefault() or Glpyh chineese etc.

ImGui::PushFont(DefaultFontRegular);

#define _S(_LITERAL) (const char*)u8##_LITERAL

ImGui::Text(_S("你好"));
ImGui::DebugTextEncoding(_S("你好"));

ImGui::PopFont();

I still see ?? and Glyph missing as shown in image attached. Project contains /Zc:char8_t /utf-8 as command line parameters (VS studio > c++ > Command Line > Additional options). The file is also saved as UTF-8 with signature.

[aDioS786]

@ocornut @PathogenDavid sorry to tag u guys in, this is closed, not sure if i should re-open another one.

[ocornut]

Thanks for details. We can keep the discussion there.

The encoding is correct (U+4F60 is 你). So the compiler setup is correct.

The problem seems to be that the glyph is not included in the font.
You can use Metrics->Fonts to browse the font contents.

I don't think seguibl.ttf contains Chinese Characters, it's a very small font (~300 KB).
You should use a font that contains Chinese Characters.

Under Windows software if you select a font that are missing some characters, it will use another font for those character (we don't do that in dear imgui).

[aDioS786]

@ocornut thanks Omar for prompt reply. But I did added "DefaultFontRegular = io.Fonts->AddFontFromFileTTF("Fonts\seguibl.ttf", 20, nullptr, io.Fonts->GetGlyphRangesChineseSimplifiedCommon());" believe that doing this in above code will merge this font into seguibl.ttf therefore I was hoping it will work.

Also I didn't understand about "You can use Metrics->Fonts to browse the font contents." can you explain this please.

[aDioS786]

I've even just tried to use https://fonts.google.com/noto/specimen/Noto+Sans+JP?preview.text=%E4%BD%A0%E5%A5%BD&preview.text_type=custom font, on website if i paste the character then it is showing correct on website.

But below code still shows ?? and Glyph "[missing]

ImFont* font = io.Fonts->AddFontDefault();
DefaultFont = io.Fonts->AddFontFromFileTTF("Fonts\NotoSansJP-Regular.otf", 20);
DefaultFontRegular = io.Fonts->AddFontFromFileTTF("Fonts\seguibl.ttf", 20);

ImGui::PushFont(DefaultFont);

#define _S(_LITERAL) (const char*)u8##_LITERAL

ImGui::Text(_S("你好"));
ImGui::DebugTextEncoding(_S("你好"));

ImGui::PopFont();

[ocornut]

Also I didn't understand about "You can use Metrics->Fonts to browse the font contents." can you explain this please.

See https://github.com/ocornut/imgui/wiki/Debug-Tools

[PathogenDavid]

DefaultFont = io.Fonts->AddFontFromFileTTF("Fonts\NotoSansJP-Regular.otf", 20);

You need to escape your backslashes. For example:

DefaultFont = io.Fonts->AddFontFromFileTTF("Fonts\\NotoSansJP-Regular.otf", 20);

Also you should be testing with debug builds so that you get errors for mistakes like this.

[aDioS786]

Debug-Tools

OH thanks yeh could be that. debug builds u referring ot same what Omar referred to? Using Debug-Tools?

[PathogenDavid]

debug builds u referring ot same what Omar referred to? Using Debug-Tools?

No, I'm referring to the debug configuration in Visual Studio. See this article on Microsoft Learn. (You should just have to change the configuration, all the default settings should be good.)

[aDioS786]

debug builds u referring ot same what Omar referred to? Using Debug-Tools?

No, I'm referring to the debug configuration in Visual Studio. See this article on Microsoft Learn. (You should just have to change the configuration, all the default settings should be good.)

Got it, yes I've been using Debug, not release. I feel kind of so badly stuck. So if i go to https://fonts.google.com/noto/specimen/Noto+Sans+JP/tester and type in above text then it works fine, means glyphs are there for this.
Then I added this font into correct folder and tried DefaultFont = io.Fonts->AddFontFromFileTTF("Fonts\NotoSansJP-Regular.otf", 20); and then push/pop font which didn't work.

After above, tried to check it via showMetrics window..

Here you can see, it's still sayiung missing glyph not sure why this thing is not working :(.

@PathogenDavid @ocornut

[aDioS786]

@ocornut @PathogenDavid Many thanks both for ur help, finally some success.

`ImGuiIO& io = ImGui::GetIO();
io.FontDefault = io.Fonts->AddFontDefault();
io.Fonts->AddFontFromFileTTF("Fonts\NotoSansJP-Regular.otf", 20, &config, io.Fonts->GetGlyphRangesJapanese());
io.Fonts->AddFontFromFileTTF("Fonts\NotoSansJP-Regular.otf", 20, &config, io.Fonts->GetGlyphRangesChineseSimplifiedCommon());
io.Fonts->AddFontFromFileTTF("Fonts\NotoSansJP-Regular.otf", 20, &config, io.Fonts->GetGlyphRangesCyrillic());

DefaultFontRegular = io.Fonts->AddFontFromFileTTF("Fonts\seguibl.ttf", 20, &config, nullptr);
io.Fonts->Build();
`

So my problem was not giving a range, my understanding is that when we have some font, and even if it contains chineese, english and other glyphs, we still need to tell the ImGui what range we need to load/use from this font, so in above e.g. it is loading all these ranges from fron the NotoSansJP font, as when I add Japanese/Chinese Range all chinese characters seems working, when i add Cyrillic then Russian characters work as well. Is this correct or I'm totally wrong with this concept.

Not I have another issue which is that DefaultFontRegular = io.Fonts->AddFontFromFileTTF("Fonts\seguibl.ttf", 20, &config, nullptr); font is Bold, as soon I add it to merge into existing config the bold goes away. Assume I need to give it some range? As if i remove the &config from it, it starts working normally. So not sure how I can get it work even having config passed to it, which is to merge with existing fonts.

Thanks

[yippeesoft]

add jp.otf/yahei.ttf ,still shows as ???