Normative: remove tables of Unicode property values and aliases #2649

michaelficarra · 2022-02-03T00:11:16Z

Following the acceptance of L2/22-029, Proposal to guarantee stability of spelling of property names, values, and aliases in UCD, we no longer need to keep these tables.

mathiasbynens

This is great! 🥳

What’s the timeline on getting the accepted proposal reflected “on paper” in the Unicode Standard? We don’t necessarily need to wait for that to happen (given the consensus) but it’d be useful to know.

michaelficarra · 2022-02-03T15:31:31Z

spec.html

          <emu-note>
            <p>For example, `Xpeo` and `Old_Persian` are valid `Script_Extensions` values, but `xpeo` and `Old Persian` aren't.</p>
          </emu-note>
          <emu-note>
            <p>This algorithm differs from <a href="https://unicode.org/reports/tr44/#Matching_Symbolic">the matching rules for symbolic values listed in UAX44</a>: case, <emu-xref href="#sec-white-space">white space</emu-xref>, U+002D (HYPHEN-MINUS), and U+005F (LOW LINE) are not ignored, and the `Is` prefix is not supported.</p>
          </emu-note>
          <emu-note>
-            <p>The spellings of entries in these tables (including casing) were chosen to match the first occurrence of each property in the files <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyAliases.txt"><code>PropertyAliases.txt</code></a> and <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a> in the Unicode Character Database at the time each entry was added to this specification. However, because the precise spellings in those files are not guaranteed to be stable, implementations are required to follow this table rather than those files.</p>
+            <p>The spellings of entries in these tables (including casing) were chosen to match the first occurrence of each property in the file <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyAliases.txt"><code>PropertyAliases.txt</code></a> in the Unicode Character Database at the time each entry was added to this specification. However, because the precise spellings in those files are not guaranteed to be stable, implementations are required to follow this table rather than those files.</p>


Is this note even needed now?

Maybe. ES still needs a list of properties, since it only supports an explicit subset.

Suggested change

The spellings of entries in these tables (including casing) were chosen to match the first occurrence of each property in the file <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyAliases.txt"><code>PropertyAliases.txt</code></a> in the Unicode Character Database at the time each entry was added to this specification. However, because the precise spellings in those files are not guaranteed to be stable, implementations are required to follow this table rather than those files.

The spellings of entries in these tables (including casing) were chosen to match the historically first occurrence of each property in the file <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyAliases.txt"><code>PropertyAliases.txt</code></a> in the Unicode Character Database at the time each entry was added to this specification. Note that the precise spellings in those files are guaranteed to be stable. Additional aliases might be added in future versions of Unicode.

michaelficarra · 2022-02-03T16:03:47Z

Not sure if we need all the "Unicode property" clarification or if we could just use "property" in most places. I guess same goes for existing usage of "Unicode code point". The qualifier mostly seems unnecessary.

michaelficarra · 2022-02-03T16:04:53Z

What’s the timeline on getting the accepted proposal reflected “on paper” in the Unicode Standard?

Not sure. @markusicu could probably answer. I was just going to let this PR sit until it happened, unless the committee asks for it to be merged sooner when I ask for consensus.

markusicu · 2022-02-04T21:52:07Z

What’s the timeline on getting the accepted proposal reflected “on paper” in the Unicode Standard?

Not sure. @markusicu could probably answer. I was just going to let this PR sit until it happened, unless the committee asks for it to be merged sooner when I ask for consensus.

The new policy has been approved by the Unicode Technical Committee and by the Unicode executive officers.
I don't know when it will be published on the website.

markusicu · 2022-02-04T21:56:22Z

spec.html

@@ -34530,13 +34530,13 @@ <h1>Static Semantics: Early Errors</h1>
            It is a Syntax Error if the List of Unicode code points that is SourceText of |UnicodePropertyName| is not identical to a List of Unicode code points that is a Unicode property name or property alias listed in the &ldquo;Property name and aliases&rdquo; column of <emu-xref href="#table-nonbinary-unicode-properties"></emu-xref>.
          </li>
          <li>
-            It is a Syntax Error if the List of Unicode code points that is SourceText of |UnicodePropertyValue| is not identical to a List of Unicode code points that is a value or value alias for the Unicode property or property alias given by SourceText of |UnicodePropertyName| listed in the &ldquo;Property value and aliases&rdquo; column of the corresponding tables <emu-xref href="#table-unicode-general-category-values"></emu-xref> or <emu-xref href="#table-unicode-script-values"></emu-xref>.
+            It is a Syntax Error if the List of Unicode code points that is SourceText of |UnicodePropertyValue| is not identical to a property value or property value alias for the Unicode property or property alias given by SourceText of |UnicodePropertyName| listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>.


A UnicodePropertyValue should only match a Unicode property value alias, right? Not also a Unicode property alias?
Thus

Suggested change

It is a Syntax Error if the List of Unicode code points that is SourceText of |UnicodePropertyValue| is not identical to a property value or property value alias for the Unicode property or property alias given by SourceText of |UnicodePropertyName| listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>.

It is a Syntax Error if the List of Unicode code points that is SourceText of |UnicodePropertyValue| is not identical to a property value alias for the Unicode property or property alias given by SourceText of |UnicodePropertyName| listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>.

?

I'm confused, where does it say Unicode property alias?

Hm, sorry, I think I misread "a property value or property value alias" for "a property alias or property value alias".
You are right, it just says "value or value alias". So not wrong, just redundant: In Unicode parlance, all of these are "aliases". The "value" is the logical thing, and the "aliases" are the symbolic strings for the thing. Thus I think my suggestion is useful despite my brain fart :-}

Example: https://www.unicode.org/reports/tr44/#Property_Value_Aliases

In PropertyValueAliases.txt, the first field contains the abbreviated alias for a Unicode property, the second field specifies an abbreviated symbolic name for a value of that property, and the third field specifies the long symbolic name for that value of that property. These are the preferred aliases. Additional aliases for some property values may be specified in the fourth or subsequent fields.

markusicu · 2022-02-04T21:58:11Z

spec.html

          </li>
        </ul>
        <emu-grammar>UnicodePropertyValueExpression :: LoneUnicodePropertyNameOrValue</emu-grammar>
        <ul>
          <li>
-            It is a Syntax Error if the List of Unicode code points that is SourceText of |LoneUnicodePropertyNameOrValue| is not identical to a List of Unicode code points that is a Unicode general category or general category alias listed in the &ldquo;Property value and aliases&rdquo; column of <emu-xref href="#table-unicode-general-category-values"></emu-xref>, nor a binary property or binary property alias listed in the &ldquo;Property name and aliases&rdquo; column of <emu-xref href="#table-binary-unicode-properties"></emu-xref>.
+            It is a Syntax Error if the List of Unicode code points that is SourceText of |LoneUnicodePropertyNameOrValue| is not identical to a Unicode property value or property value alias for the General_Category (gc) property listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>, nor a binary property or binary property alias listed in the &ldquo;Property name and aliases&rdquo; column of <emu-xref href="#table-binary-unicode-properties"></emu-xref>.


ditto?

Suggested change

It is a Syntax Error if the List of Unicode code points that is SourceText of |LoneUnicodePropertyNameOrValue| is not identical to a Unicode property value or property value alias for the General_Category (gc) property listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>, nor a binary property or binary property alias listed in the “Property name and aliases” column of <emu-xref href="#table-binary-unicode-properties"></emu-xref>.

It is a Syntax Error if the List of Unicode code points that is SourceText of |LoneUnicodePropertyNameOrValue| is not identical to a Unicode property value alias for the General_Category (gc) property listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>, nor a binary property name or binary property alias listed in the “Property name and aliases” column of <emu-xref href="#table-binary-unicode-properties"></emu-xref>.

markusicu · 2022-02-04T21:59:46Z

spec.html

@@ -35656,7 +35656,7 @@ <h1>Runtime Semantics: CompileToCharSet</h1>
        <emu-grammar>UnicodePropertyValueExpression :: LoneUnicodePropertyNameOrValue</emu-grammar>
        <emu-alg>
          1. Let _s_ be SourceText of |LoneUnicodePropertyNameOrValue|.
-          1. If ! UnicodeMatchPropertyValue(`General_Category`, _s_) is identical to a List of Unicode code points that is the name of a Unicode general category or general category alias listed in the &ldquo;Property value and aliases&rdquo; column of <emu-xref href="#table-unicode-general-category-values"></emu-xref>, then
+          1. If ! UnicodeMatchPropertyValue(`General_Category`, _s_) is identical to a Unicode property value or property value alias for the General_Category (gc) property listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>, then


Suggested change

1. If ! UnicodeMatchPropertyValue(`General_Category`, _s_) is identical to a Unicode property value or property value alias for the General_Category (gc) property listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>, then

1. If ! UnicodeMatchPropertyValue(`General_Category`, _s_) is identical to a Unicode property value alias for the General_Category (gc) property listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>, then

markusicu · 2022-02-04T22:02:29Z

spec.html

            1. Let _value_ be the canonical property value of _v_ as given in the &ldquo;Canonical property value&rdquo; column of the corresponding row.
            1. Return the List of Unicode code points _value_.
          </emu-alg>
-          <p>Implementations must support the Unicode property value names and aliases listed in <emu-xref href="#table-unicode-general-category-values"></emu-xref> and <emu-xref href="#table-unicode-script-values"></emu-xref>. To ensure interoperability, implementations must not support any other property value names or aliases.</p>
+          <p>Implementations must support the Unicode property values and property value aliases listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a> for the properties listed in <emu-xref href="#table-nonbinary-unicode-properties"></emu-xref>. To ensure interoperability, implementations must not support any other property values or property value aliases.</p>


You are pointing to the "lastest" Unicode Character Database, which makes this spec "evergreen". However, as a "must support", should you provide a little leniency for recent versions, or the versions as of the release of the ES implementation?

We already normatively refer to the latest version of the Unicode standard (for things like whitespace, ID_Start, etc). From the spec,

Additionally, ECMAScript 2017 mandated always using the latest version of the Unicode standard.

For context, this was the relevant PR: #620

markusicu · 2022-02-04T22:06:22Z

spec.html

          <emu-note>
            <p>For example, `Xpeo` and `Old_Persian` are valid `Script_Extensions` values, but `xpeo` and `Old Persian` aren't.</p>
          </emu-note>
          <emu-note>
            <p>This algorithm differs from <a href="https://unicode.org/reports/tr44/#Matching_Symbolic">the matching rules for symbolic values listed in UAX44</a>: case, <emu-xref href="#sec-white-space">white space</emu-xref>, U+002D (HYPHEN-MINUS), and U+005F (LOW LINE) are not ignored, and the `Is` prefix is not supported.</p>
          </emu-note>
          <emu-note>
-            <p>The spellings of entries in these tables (including casing) were chosen to match the first occurrence of each property in the files <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyAliases.txt"><code>PropertyAliases.txt</code></a> and <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a> in the Unicode Character Database at the time each entry was added to this specification. However, because the precise spellings in those files are not guaranteed to be stable, implementations are required to follow this table rather than those files.</p>
+            <p>The spellings of entries in these tables (including casing) were chosen to match the first occurrence of each property in the file <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyAliases.txt"><code>PropertyAliases.txt</code></a> in the Unicode Character Database at the time each entry was added to this specification. However, because the precise spellings in those files are not guaranteed to be stable, implementations are required to follow this table rather than those files.</p>


Maybe. ES still needs a list of properties, since it only supports an explicit subset.

Suggested change

The spellings of entries in these tables (including casing) were chosen to match the first occurrence of each property in the file <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyAliases.txt"><code>PropertyAliases.txt</code></a> in the Unicode Character Database at the time each entry was added to this specification. However, because the precise spellings in those files are not guaranteed to be stable, implementations are required to follow this table rather than those files.

The spellings of entries in these tables (including casing) were chosen to match the historically first occurrence of each property in the file <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyAliases.txt"><code>PropertyAliases.txt</code></a> in the Unicode Character Database at the time each entry was added to this specification. Note that the precise spellings in those files are guaranteed to be stable. Additional aliases might be added in future versions of Unicode.

markusicu · 2022-02-04T22:08:14Z

spec.html

@@ -34530,13 +34530,13 @@ <h1>Static Semantics: Early Errors</h1>
            It is a Syntax Error if the List of Unicode code points that is SourceText of |UnicodePropertyName| is not identical to a List of Unicode code points that is a Unicode property name or property alias listed in the &ldquo;Property name and aliases&rdquo; column of <emu-xref href="#table-nonbinary-unicode-properties"></emu-xref>.


Note: You could simplify the tables of properties: List only one name for each property, no further aliases, and just point to PropertyAliases.txt for aliases.

markusicu · 2022-03-25T17:21:01Z

What’s the timeline on getting the accepted proposal reflected “on paper” in the Unicode Standard?

Not sure. @markusicu could probably answer. I was just going to let this PR sit until it happened, unless the committee asks for it to be merged sooner when I ask for consensus.

The new policy has been approved by the Unicode Technical Committee and by the Unicode executive officers. I don't know when it will be published on the website.

The updated policy has been published today:
https://www.unicode.org/policies/stability_policy.html#Alias_Stability
...
Property aliases, once defined in PropertyAliases.txt, will never be removed, nor will their precise spelling be changed.

Property value aliases, once defined in PropertyValueAliases.txt, will never be removed, nor will their precise spelling be changed.
...

spec.html

michaelficarra · 2022-08-04T01:59:35Z

@bakkot rebased and addressed comment

spec.html

michaelficarra added normative change Affects behavior required to correctly evaluate some ECMAScript source text needs consensus This needs committee consensus before it can be eligible to be merged. labels Feb 3, 2022

michaelficarra requested review from mathiasbynens, a team and markusicu February 3, 2022 00:11

bakkot mentioned this pull request Feb 3, 2022

Normative: Add RegExp v flag with set notation and properties of strings #2418

Merged

mathiasbynens approved these changes Feb 3, 2022

View reviewed changes

michaelficarra commented Feb 3, 2022

View reviewed changes

markusicu reviewed Feb 4, 2022

View reviewed changes

michaelficarra added the editor call to be discussed in the next editor call label Feb 5, 2022

michaelficarra added a commit to tc39/agendas that referenced this pull request Feb 7, 2022

add tc39/ecma262#2649 topic

77b3d0a

michaelficarra removed the editor call to be discussed in the next editor call label Feb 16, 2022

michaelficarra added has consensus This has committee consensus. and removed needs consensus This needs committee consensus before it can be eligible to be merged. labels Mar 28, 2022

michaelficarra added editor call to be discussed in the next editor call and removed editor call to be discussed in the next editor call labels Aug 3, 2022

bakkot reviewed Aug 3, 2022

View reviewed changes

spec.html Outdated Show resolved Hide resolved

michaelficarra force-pushed the stable-Unicode-property-spellings branch from 3a55d88 to 5f86037 Compare August 4, 2022 01:59

michaelficarra requested a review from bakkot August 4, 2022 01:59

bakkot reviewed Aug 8, 2022

View reviewed changes

spec.html Outdated Show resolved Hide resolved

bakkot approved these changes Aug 29, 2022

View reviewed changes

michaelficarra added the ready to merge Editors believe this PR needs no further reviews, and is ready to land. label Aug 31, 2022

Normative: remove tables of Unicode property values and aliases (#2649)

3c29c06

ljharb force-pushed the stable-Unicode-property-spellings branch from e7496c3 to 3c29c06 Compare September 1, 2022 17:04

ljharb merged commit 3c29c06 into main Sep 1, 2022

ljharb deleted the stable-Unicode-property-spellings branch September 1, 2022 17:20

This was referenced Mar 13, 2023

Bug Script=Unknown eslint-community/regexpp#57

Closed

feat: support Unicode properties added in ES2023 eslint-community/regexpp#59

Merged

ota-meshi mentioned this pull request May 20, 2023

Add remaining Unicode properties for ES2023 acornjs/acorn#1217

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Normative: remove tables of Unicode property values and aliases #2649

Normative: remove tables of Unicode property values and aliases #2649

michaelficarra commented Feb 3, 2022

mathiasbynens left a comment

michaelficarra Feb 3, 2022

markusicu Feb 4, 2022

michaelficarra commented Feb 3, 2022

michaelficarra commented Feb 3, 2022

markusicu commented Feb 4, 2022

markusicu Feb 4, 2022

michaelficarra Feb 4, 2022

markusicu Feb 5, 2022

markusicu Feb 4, 2022

markusicu Feb 4, 2022

markusicu Feb 4, 2022

michaelficarra Feb 4, 2022

mathiasbynens Mar 24, 2022

markusicu Feb 4, 2022

markusicu Feb 4, 2022

markusicu commented Mar 25, 2022

michaelficarra commented Aug 4, 2022

	It is a Syntax Error if the List of Unicode code points that is SourceText of \|UnicodePropertyValue\| is not identical to a property value or property value alias for the Unicode property or property alias given by SourceText of \|UnicodePropertyName\| listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>.
	It is a Syntax Error if the List of Unicode code points that is SourceText of \|UnicodePropertyValue\| is not identical to a property value alias for the Unicode property or property alias given by SourceText of \|UnicodePropertyName\| listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>.

	1. If ! UnicodeMatchPropertyValue(`General_Category`, _s_) is identical to a Unicode property value or property value alias for the General_Category (gc) property listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>, then
	1. If ! UnicodeMatchPropertyValue(`General_Category`, _s_) is identical to a Unicode property value alias for the General_Category (gc) property listed in <a href="https://unicode.org/Public/UCD/latest/ucd/PropertyValueAliases.txt"><code>PropertyValueAliases.txt</code></a>, then

		@@ -34530,13 +34530,13 @@ <h1>Static Semantics: Early Errors</h1>
		It is a Syntax Error if the List of Unicode code points that is SourceText of \|UnicodePropertyName\| is not identical to a List of Unicode code points that is a Unicode property name or property alias listed in the “Property name and aliases” column of <emu-xref href="#table-nonbinary-unicode-properties"></emu-xref>.

Normative: remove tables of Unicode property values and aliases #2649

Normative: remove tables of Unicode property values and aliases #2649

Conversation

michaelficarra commented Feb 3, 2022

mathiasbynens left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michaelficarra commented Feb 3, 2022

michaelficarra commented Feb 3, 2022

markusicu commented Feb 4, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markusicu commented Mar 25, 2022

michaelficarra commented Aug 4, 2022