Wrong font picked up for optical size font

[zhouyan]

I just noticed that this is a duplicate with #356 . However I hope the details below are useful

I noticed that fontspec is picking up wrong fonts in some situations, when there are multiple optical sized fonts. I traced down the issues to luaotfload. And created the following minimal plain tex example,

\input luaotfload.sty
\font\aseven=ArnoPro at 7pt
\font\aten=ArnoPro at 10pt
\font\atwenty=ArnoPro at 20pt
\font\abiseven=ArnoPro/BI at 7pt
\font\abiten=ArnoPro/BI at 10pt
\font\abitwenty=ArnoPro/BI at 20pt
\font\cseven=CronosPro at 7pt
\font\cten=CronosPro at 10pt
\font\ctwenty=CronosPro at 20pt
\font\cbiseven=CronosPro/BI at 7pt
\font\cbiten=CronosPro/BI at 10pt
\font\cbitwenty=CronosPro/BI at 20pt

\def\fontname{%
  {%
    \directlua{
      local i = font.current()
      local f = font.getfont(i)
      local n = f.fullname
      tex.sprint(n)
    }%
  }%
}

\aseven\fontname\par
\aten\fontname\par
\atwenty\fontname\par
\abiseven\fontname\par
\abiten\fontname\par
\abitwenty\fontname\par
\cseven\fontname\par
\cten\fontname\par
\ctwenty\fontname\par
\cbiseven\fontname\par
\cbiten\fontname\par
\cbitwenty\fontname\par

\bye

In the output, two out of the last three fonts was wrong. They should be CronosPro-BoldCaptIt, CronosPro-BoldIt, and CronosPro-BoldSubhIt

I included the selection for Arno Pro and up shape of Cronos Pro for comparision. It appears that,

1. luaotfload is able to choose Arno Pro correclty, for both up shape, and bold italics. (italics, and bold also works fine)
2. It is choosing Cronos Pro correclty when using up shape.
3. It uses only CronosPro-BoldIt for all sizes of bold italics (It also get wrong for Display size bold).

So I looked up the luaotfload-names.lua.gz file and here is a few thing I noticed that are different when it is choosing the right font and wrong ones (I also did some more expensive experiments with all Adobe optical fonts).

1. It works fine when the font names are like BoldIt, or BoldItalics, or BoldItSubh etc.
2. It is not OK for the font be named as BoldSubhIt, instead of BoldItSubh.

Further, it seems the problem is not with the the main package, instead of it is with luaotfload-tool

Below is the entry for cronospro generated by luaotfload-tool

["cronospro"]={
     ["b"]={
      { 8, 8, 6, 3250 },
      { 8, 8, 6, 3251 },
      { 11, 14, 8, 3249 },
      { 18, 24, 14, 3255 },
      { 18, 24, 14, 3256 },
      { 72, 72, 24, 3253 },
      { 72, 72, 24, 3252 },
     },
     ["bi"]={
      { 11, 14, 8, 3254 },
     },
     ["i"]={
      { 11, 14, 8, 3261 },
     },
     ["r"]={
      { 8, 8, 6, 3257 },
      { 11, 14, 8, 3262 },
      { 18, 24, 14, 3263 },
      { 72, 72, 24, 3259 },
     },
    },

Some entries that should belong to the table "bi" is in "b" instead. Similary for the italic group. After manually fix this table to the following,

    ["cronospro"]={
     ["b"]={
      { 8, 8, 6, 3250 },
      { 11, 14, 8, 3249 },
      { 18, 24, 14, 3255 },
      { 72, 72, 24, 3253 },
     },
     ["bi"]={
      { 8, 8, 6, 3251 },
      { 11, 14, 8, 3254 },
      { 18, 24, 14, 3256 },
      { 72, 72, 24, 3252 },
     },
     ["i"]={
      { 8, 8, 6, 3258 },
      { 11, 14, 8, 3261 },
      { 18, 24, 14, 3264 },
      { 72, 72, 24, 3260 },
     },
     ["r"]={
      { 8, 8, 6, 3257 },
      { 11, 14, 8, 3262 },
      { 18, 24, 14, 3263 },
      { 72, 72, 24, 3259 },
     },
    },

The test file at the beginning of this issue is working correctly.

I am not able to determine the where exactly did luaotfload-tool get it wrong. There are other related issues, such as seemly randomly chosen Light or Medium weights for Kepler Std, etc. I think even if we do find a fix for these issues, it is not future proof. The naming of fonts are just so inconsistent. Even within Adobe fonts, as we see in this issue, there are different conventions. In summary, I don't think this is an issue that worth the effort to fix.

Instead I suggest a public interface for users to deal with these edge cases directly. For example, in luaotfload-tool, after generating the table, but before write it to the file, it will read a user supplied lua script if it exists, call a function, say patch_name_table within it if it exist, and pass the table to it. The user defined patch_name_table will allow the user to do whatever final touch they want.

[phi-gamma]

Hi,

thanks for the report.

···<date: 2016-06-12, Sunday>···<from: Yan Zhou>···

I noticed that fontspec is picking up wrong fonts in some
situations, when there are multiple optical sized fonts. I traced
down the issues to luaotfload.

Please wait for the next bugfix release; a number of issues
regarding font selection have been addressed in the meantime.

I included the selection for Arno Pro and up shape of Cronos Pro for comparision. It appears that,

1. luaotfload is able to choose Arno Pro correclty, for both up shape, and bold italics. (italics, and bold also works fine)
2. It is choosing Cronos Pro correclty when using up shape.
3. It uses only CronosPro-BoldIt for all sizes of bold italics (It also get wrong for Display size bold).

If this doesn’t vanish with the upcoming release, it’d be helpful
if you could reproduce the issue with a free font that can be
tested independently.

I am not able to determine the where exactly did
luaotfload-tool get it wrong.

Names table, as usual. Almost all fonts are broken in this
aspect, and the standard is crap too.

There are other related issues, such as seemly randomly chosen
Light or Medium weights for Kepler Std, etc. I think even if
we do find a fix for these issues, it is not future proof. The
naming of fonts are just so inconsistent. Even within Adobe
fonts, as we see in this issue, there are different conventions.
In summary, I don't think this is an issue that worth the effort
to fix.

Names even vary over different versions of the same font and are
inconsistent between the two (Mac and Win) names tables.

One way to address the issue would be to extend font selection
conceptually to something like what xfontsel does. That would
mean that in addition to the mere names other criteria like width
and numerical weight can be specified to narrow down the
selection. That way one could overcome the overly simplistic
“font name { + weight } { + obliqueness }” scheme. But since none
of the existing extensions to the font specifications in
Luaotfload appear to be used downstream, I’m reluctant to invest
any effort in this direction.

Instead I suggest a public interface for users to deal with
these edge cases directly. For example, in luaotfload-tool,
after generating the table, but before write it to the file, it
will read a user supplied lua script if it exists, call a
function, say patch_name_table within it if it exist, and pass
the table to it. The user defined patch_name_table will allow
the user to do whatever final touch they want.

Nah, just use filenames directly, maybe in conjunction with a
font definition file in the eu2 fashion. The test repo has an
example for Arno Pro:

https://bitbucket.org/phg/lua-la-tex-tests/src/ce4fc8dddb562d3f938e5c96c473bc77fda2aa1c/eu2arnopro.fd

That’s the most adequate means of setting up fonts for Latex that
I’m aware of, similar to Context’ś typescripts.

Best,
Philipp

[zhouyan]

Just to add, I tested my examples with Github master version before submitting the issue. I will try to find a free font example later

Another way I can think of, and perhaps less complicated as allowing patching the table (which will be quite painful considering the index values are hashing values, they can change between updating even without change anything on the system) is to allow a configure file similar to the blacklist file
For example:

KeplerStd-Regular: r
KeplerStd-Medium: nil
CronosPro-BoldCaptIt: bi
SanvitoPro-Regular: r, it

The first explicitly says Kepler regular belongs to table "r", the second exclude the corresponding font from any four of them. That third explicitly says it belongs to table "bi" and thus preventing it being included in "b". The last says we want to include it in both r and it tables.

[phi-gamma]

The new release is out, please check whether this improves your issue.

[zhouyan]

Thanks for the update. I haven't got time to do more extensive testing. But the issue described still persists. And in some cases it is worse. For example, at the end of this comment is the contents of families.system.otf.keplerstd table. I added some comments to make it clear the font selected (omitting the KeplerStd- prefix) for each entry. In this case, the "b" table will lead to incorrect selection of fonts for every optical size. In fact, it will select an italic bold fonts with wrong widths most of the time. The bi and i tables are better, they now have correct entries. The "r" table will select incorrect fonts for larger sizes.

Kepler is a particularly difficult case, there are 4 widths, 5 weights, 4 sizes, 2 styles, in total 169 fonts in the family.

My current workaround is to manual edit luaotfload-names.lua.gz file, which involves a lot going forth and back to identify the correct ID of the 16 essential fonts after running luaotfload-tool, and delete the luc file, and cache files. This makes simple usage such as

\setmainfont{Kepler Std}

works smoothly. And if I need a specific style, say extended bold italics, I use \font to select the particular font. This is OK since it is going to be used only in a few places or systematically used through the documents for some typographical elements.

I can't find a free font that has such a broad range of styles to "stress test" the package. I still think it is best if things are kept simple, work for most simple cases. And with an easy interface for dealing with these edges, such as the configuration file I proposed earlier.

The DelcareFontShape approach is nice. But still a bit tedious. And I don't know how it interacts with fontspec (I will find some time to do more testing on this before making any informed comments).

Manually editing the database is certainly tedious. I wrote a script to automate the process, which is used everytime I run luaotfload-tool. However, it will only work for myself. Perhaps I can write one to support all Adobe optical fonts, and some others with extended family of styles, for more general usage. The other drawback of editing it is that I need to delete the luc file, which will not be regenerated when I compile a document. That seems OK. The slow down is only slightly noticeable.

If you think the idea of a configuration file, which will be read in and put the fonts in it in the appropriate tables before automatically selected ones, I will see if I can create a patch and send a pull request. But before that, I need to get myself familiar with the source first and can't promise anything in the near future.

In any case, luaotfload has worked very well for me and I believe for most other users. The issues I described here can be considered only as edge cases. And I don''t think it worth the effort to fix them in the codebase. If anything, it might make things worse. As a programmer, I understand how tangled these things are, and how difficult to build a fool proof system. So, I want to thank the develops for the hard work.

-- Naming convensions: <Weight><Width><Shape><Size>
-- Weight
--      (Normal)
--      Light
--      Medium
--      Bold
--      Black
-- Width
--      (Regular)
--      Cn:  Condensed
--      Scn: Semicondensed
--      Ext: Extended
-- Shape
--      (Upright)
--      It: Italic
-- Size
--      (Text)
--      Capt: Caption
--      Subh: Subhead
--      Disp: Display
["keplerstd"]={
  ["b"]={
    { 8, 8.9, 6, 2465 },     --   BoldExtItCapt
    { 8, 8.9, 6, 2474 },     --   BoldScnCapt
    { 8, 8.9, 6, 2477 },     --   BoldScnItCapt
    { 8, 8.9, 6, 2462 },     --   BoldExtCapt
    { 8, 8.9, 6, 2455 },     -- * BoldCapt
    { 11, 13.9, 8.9, 2464 }, --   BoldExtIt
    { 11, 13.9, 8.9, 2473 }, --   BoldScn
    { 11, 13.9, 8.9, 2476 }, --   BoldScnIt
    { 11, 13.9, 8.9, 2454 }, -- * Bold
    { 11, 13.9, 8.9, 2461 }, --   BoldExt
    { 18, 23, 13.9, 2459 },  --   BoldCnSubh
    { 18, 23, 13.9, 2481 },  -- * BoldSubh
    { 18, 23, 13.9, 2479 },  --   BoldScnItSubh
    { 18, 23, 13.9, 2468 },  --   BoldExtSubh
    { 18, 23, 13.9, 2458 },  --   BoldCnItSubh
    { 18, 23, 13.9, 2467 },  --   BoldExtItSubh
    { 18, 23, 13.9, 2480 },  --   BoldScnSubh
    { 72, 72, 23, 2478 },    --   BoldScnItDisp
    { 72, 72, 23, 2460 },    -- * BoldDisp
    { 72, 72, 23, 2466 },    --   BoldExtItDisp
    { 72, 72, 23, 2456 },    --   BoldCnDisp
    { 72, 72, 23, 2475 },    --   BoldScnDisp
    { 72, 72, 23, 2457 },    --   BoldCnItDisp
    { 72, 72, 23, 2463 },    --   BoldExtDisp
  },
  ["bi"]={
    { 8, 8.9, 6, 2470 },     -- * BoldItCapt
    { 11, 13.9, 8.9, 2469 }, -- * BoldIt
    { 18, 23, 13.9, 2472 },  -- * BoldItSubh
    { 72, 72, 23, 2471 },    -- * BoldItDisp
  },
  ["i"]={
    { 8, 8.9, 6, 2497 },     -- * ItCapt
    { 11, 13.9, 8.9, 2496 }, -- * Italic
    { 18, 23, 13.9, 2499 },  -- * ItSubh
    { 72, 72, 23, 2498 },    -- * ItDisp
  },
  ["r"]={
    { 8, 8.9, 6, 2482 },     -- * Capt
    { 8, 8.9, 6, 2548 },     --   MediumScnCapt
    { 8, 8.9, 6, 2520 },     --   LightScnCapt
    { 11, 13.9, 8.9, 2556 }, -- * Regular
    { 11, 13.9, 8.9, 2519 }, --   LightScn
    { 11, 13.9, 8.9, 2547 }, --   MediumScn
    { 18, 23, 13.9, 2554 },  --   MediumScnSubh
    { 18, 23, 13.9, 2526 },  --   LightScnSubh
    { 18, 23, 13.9, 2593 },  -- * Subh
    { 72, 72, 23, 2521 },    --   LightScnDisp
    { 72, 72, 23, 2487 },    -- * Disp
    { 72, 72, 23, 2549 },    --   MediumScnDisp
  },
},

[zhouyan]

Another reason for allowing easier patching of the families table is that, many fonts come with an extensive range of width and weights, which are meant to be used at different optical sizes. It is impossible for luaotfload to properly setup the four tables because there are no size range informations in the font at all. However, for experienced users, they often select different weights based on the text size. And allowing easier setup for this purpose can be very handy. For example, Myriad Pro comes with about forty styles, and below is the table generated by luaotfload. I commented on the fonts that's going to be selected for each of the four basic styles. The condensed version is selected as default upright font while the regular version is clearly much more appropriate.

    ["myriadpro"]={
     ["b"]={
      { 11, 0, 0, 2159 }, -- Bold
      -- a few other entries
     },
     ["bi"]={
      { 11, 0, 0, 2162 }, -- BoldIt
     },
     ["i"]={
      { 11, 0, 0, 2169 }, -- It
     },
     ["r"]={
      { 11, 0, 0, 2178 }, -- Condensed
     -- some other entries
     },
    },

This particular example is produced on CentOS, while Mac OS X, with the same font collection, the choice was regular.

[zhouyan]

There is actually one free fonts with extensive optical ranges, the Computer/Latin Modern fonts. I rarely use it nowadays and overlooked it. After updating to the most recent release of luaotfload, I run luaotfload-tool -u -f, and examined the database. Similar issues exists for Latin Modern as well. Below is the table generated for lmroman, lmsans and lmmono. In all three cases, the table of italics is missing. This is not a big issue for the mono font, since there is only one slanted mono (lmmono10-italic). But for the roman and sans fonts, there are multiple optical sized ones. And they are missing.

["latinmodernmono"]={
  -- Missing italic table, lmmono10-italic
  ["r"]={
    { 8, 8.5, 4, 812 },
    { 9, 9.5, 8.5, 813 },
    { 10, 11, 9.5, 810 },
    { 12, 24, 11, 811 },
  },
},

["latinmodernroman"]={
  ["b"]={
    { 5, 5.5, 3, 837 },
    { 6, 6.5, 5.5, 839 },
    { 7, 7.5, 6.5, 841 },
    { 8, 8.5, 7.5, 844 },
    { 9, 9.5, 8.5, 847 },
    { 10, 11, 9.5, 829 },
    { 12, 24, 11, 833 },
  },
  ["bi"]={
    { 10, 20, 5, 830 },
  },
  -- Missing italic table, lmroman{7,8,9,10,12}-italic
  -- Only lmroman10-italic will be used in documents
  ["r"]={
    { 5, 5.5, 3, 838 },
    { 6, 6.5, 5.5, 840 },
    { 7, 7.5, 6.5, 843 },
    { 8, 8.5, 7.5, 846 },
    { 9, 9.5, 8.5, 849 },
    { 10, 11, 9.5, 832 },
    { 12, 14, 11, 835 },
    { 17.2, 24, 14, 836 },
  },
},

["latinmodernsans"]={
  ["b"]={
    { 10, 20, 5, 863 },
  },
  ["bi"]={
    { 10, 20, 5, 864 },
  },
  -- Missing italic table, lmsans{8,9,10,12,17}-oblique
  -- Only lmsans10-oblique will be used in documents
  ["r"]={
    { 8, 8.5, 4, 872 },
    { 9, 9.5, 8.5, 874 },
    { 10, 11, 9.5, 866 },
    { 12, 14, 11, 868 },
    { 17.2, 24, 14, 870 },
  },
},

[u-fischer]

I checked in a current luaotfload the latinmodernmono etc tables from the last example and the italic entries are there. So I assume that the problem has been resolved. If not open a new issue at https://github.com/u-fischer/luaotfload/.