Arabic

Arabic (language)

Second-Level Reference Label Generation Rules, 25 October 2024 Current

Download lgr-second-level-arabic-language-25oct24-en.xml (60.3kb)

Summary

Version3 — Second Level Reference LGR
Date
Tag ar (Arabic)
Unicode11.0.0
Source filelgr-second-level-arabic-language-25oct24-en.xml (60.3kb)

This ruleset is integrated into the Common (integrated set) ruleset of this release.

Description

INSTRUCTIONS

  • These instructions cover how to adopt an LGR based on this reference LGR for a given zone and how to prepare the file for deposit in the IANA Repository of IDN Practices.
  • As described the IANA procedure (https://www.iana.org/help/idn-repository-procedure) an LGR MUST contain the following elements in its header:
    • Script or Language Designator (see below for guidance)
    • Version Number (this must increase with each amendment to the LGR, even if the updates are limited to the header itself)
    • Effective Date (the date at which the policy becomes applicable in operational use)
    • Registry Contact Details (contact name, email address, and/or phone number)
  • The following information is optional:
    • Document creation date
    • Applicable Domain(s)
    • Changes made to the Reference LGR before adopting

Please add or modify the following items in the XML source code for this file before depositing the document in the IANA Repository. (https://www.iana.org/domains/idn-tables)

Meta Data

Note: version numbers start at 1. RFC 7940 recommends using simple integers. The version comment is optional, please replace or delete the default comment. Version comments may be used by some tools as part of the page header.

<version comment="[Please replace (or delete) the optional comment]">[Please fill in version number, starting at 1]</version>

<date>[Please fill in with publication date, in YYYY-MM-DD format]</date>

<validity-start>[Please fill in effective date, in YYYY-MM-DD format]</validity-start>

Note: the scope element may be repeated, so that the same document can serve for multiple domains.

<scope type="domain">[Please provide, in ".domain" format]</scope>

Registry Contact Information:

Please fill in the Registry Contact Details.

Change History

If you made technical modifications to the LGR, please summarize them in the Change History (and also note the details in the appropriate section of the description).

PLEASE DELETE THESE INSTRUCTIONS BEFORE DEPOSITING THE DOCUMENT

Read the rest of the description Show less

Registry Contact Details

  • Contact Name: [Please fill in Contact Name]
  • Email address: [Please fill in Email address]
  • Phone Number: [Please fill in optional Phone Number]

Label Generation Rules for the Arabic Language

Overview

This document specifies a set of Label Generation Rules (LGR) for the Arabic language for the second level domain or domains identified above. The ultimate starting point for the development of this LGR can be found in the related Root Zone LGR [RZ-LGR-Arab]. Note that while it is the Arabic Script Root Zone LGR that forms the starting point, the LGR defined here covers the Arabic Language. The format of this file follows [RFC 7940]. This LGR is adapted from the “Reference LGR for the Second Level for the Arabic Language” [Ref-LGR-ar-Arab], for details, see Change History below.

For details and additional background on the Arabic script, see “Proposal for Arabic Script Root Zone LGR”, Version 3.4, 18 November 2015 [Proposal-Arabic]. Additional work was done by a TF-AIDN group to create extensions appropriate to the second level and these are incorporated in this LGR. (See “Methodology” below).

The document contains certain provisions for customization. (See “Customizing actions for International Reachability” below.)

Standalone LGR: This LGR is designed to be used in a zone that does not cater to IDNs other than those valid under this LGR. This LGR lacks features that would allow its use in the context of another LGR in the same zone, and it may contain other features incompatible with such use.

Repertoire

The repertoire is a subset of the one described in Section 3.2 of [Proposal-Arabic] and only includes the 36 code points for letters in active everyday general use in the Arabic language. The repertoire is a subset of [Unicode 11.0.0]. For details, see Section 3.2 in [Proposal-Arabic]. (The proposal cited has been adopted for the Arabic script portion of the Root Zone LGR.)

For the second level, the repertoire has been augmented with the HYPHEN-MINUS, and two sets of decimal digits:

  • European (common) digits
  • Arabic-Indic digits

for a total of 57 repertoire elements.

This LGR does not include combining marks or code point sequences. All combining marks have been excluded for these reasons:

  • First, they can significantly overproduce and would require additional rules to contain them effectively, complicating the design.
  • Second, even where they are required for some languages, they are optional for others.
  • Third, this also circumvents the issue regarding duplication between some precomposed code points and combining sequences raised by [IAB].

Any code points outside the Arabic Language repertoire that are targets for out-of-repertoire variants would be included here only if the variant is listed in this file. In this case they are identified as a reflexive (identity) variant of type “out-of-repertoire-var”. Whether or not they are listed, they do not form part of the repertoire.

Repertoire Listing: Each code point or range is tagged with the script or scripts with which the code point is used and one or more other character categories. For each repertoire element, one or more references document sufficient justification for inclusion in the repertoire; see the “References” below. For code points in the repertoire, comments identify the language using the code point.

Correlation Between Repertoire and Bidi Rules

Because this LGR contains only a subset of PVALID code points, the Bidi rules expressed in Section 2 of RFC 5893 [160] (Bidi Rule) can be simplified to two constraints:

  • Prevent digits (whether European, Arabic-Indic, or Extended-Arabic-Indic) from starting a label.
  • In any label, allow only digits from one the sets (whether European, Arabic-Indic, or Extended-Arabic-Indic).

Variants

This LGR is designed for use in zones shared by more than one language. For details, see Section 3, “Use of Multiple Reference LGRs in the Same Zone” in [Level-2-Overview]. Where appropriate, cross-language variants have been defined to mutually exclude labels from different languages that could otherwise be substituted by the users. Once a label has been delegated for one language, any of its variant labels consisting entirely of cross-language variants would be blocked. Any label containing at least one code point that is not a cross-language variant would be considered distinct. Because of that, even large numbers of defined cross-language variants generally do not lead to a high percentage of labels experiencing a collision.

Digit Variants: All Arabic digits are treated as semantic variants of the corresponding common (ASCII) digits. By transitivity, they are also semantic variants of any native digits in scripts that also include the common digits. Such variant relations are deemed to exist implicitly by transitivity but are not listed explicitly in each reference LGR. Instead, if needed, they are applied by using the Common LGR in label processing.

Because of the restriction on leading digits, most labels would contain a unique code point in addition to any digits, thus no variant labels would actually result from any pro-forma cross-script variants.

To keep digit variant sets manageable in zones where multiple scripts are present, no attempt has been made at identifying cross-script variants among digits of different numeric value or between a digit in one script and a letter in another, such as between digit zero and Latin letter ‘o’. Other mechanisms may be required to prevent homograph labels.

Variant Disposition: This LGR includes “blocked” and “allocatable” variants, assigned according to Section 4, “Final recommendation of variants for Top Level Domains (TLDs)” in [Proposal-Arabic]. These recommendations balance the desire to minimize the number of possible allocatable variants with the need to keep the definition of variants simple. See also the comments given in the listing.

For the second level, three additional variant types are added: “optionally-allocatable”, “activated”, and “optionally-activated”. They are used to improve user acceptance. The rationale for these new types is provided below. Note that some of these optional variants map to code points that are not declared as part of the repertoire. If such optional variants are enabled, the reflexive “out-of-repertoire-var” mappings for the targets of these mappings must be removed, lest the generated variants fail to validate.

Methodology for Defining Variants

In generating the variant tables, a number of concepts have been used:

  • When building a variant table, for each code point in the Arabic language repertoire, a full study was conducted across the whole Arabic script in order to identify all possible variants corresponding to the code point. This was done for more protection to the registry’s TLD-space and to minimize the overhead of re-studying when the registry decides to support more than one language from the Arabic script. Hence, conducting this variant study across the whole Arabic script provides an IDN registry with the following benefits:

    • Protection to the registry name space regardless of the supported languages.
    • Doing the work for a language one time. No need to re-study the code-point’s relationship each time a new language (or new set of code point) is supported by the registries.
    • Flexibility to add more language as they become ready without affecting currently supported languages.
    • Consistency with the principle of identifying all possible variants without neglecting or overlooking any similarities without documenting it.

  • One of the main principles for the stability of the Internet and IDNs is that the end user should be able to reach a website connected to his/her domain name regardless of location. Additionally, an end user reads and types website addresses based on his/her language alphabet and whatever is available in his/her keyboard. Therefore, in order to enforce this principle the input devices (language table) that the user may use to reach a domain name (based on the user location) should be carefully considered when defining variants. Otherwise, it may cause a reachability problem and reduce the user acceptance. For example, if someone registered the domain name “مكة” (all characters from the Arabic language) and a user try to reach the website connected to this domain name from an Internet café or airport, say, in Pakistan. He/she will not be able to reach that website unless if the variant “مکۃ” (Urdu variant) is already allocated and activated. Thus, variants need to be studied from both similarity point of view (by language community) and reachability point of view (based on input devices used by other language communities).

  • Consistency is a very important concept in generating variants. Regardless of the selected applied-for label the list of generated allocatable variants should be the same. As we are dealing with normal users at the SLD, their 1st choice (applied-for) label might not be the one that will be used by the internet community. Therefore, a registry should provide the registrant the ability to “correct” his/her choice if he/she was not successful with the first try. For example, the word “Internet” is written in most Arab countries in North Africa as (أنترنت) while it is written in other Arab countries as (إنترنت), while end users often write it as (انترنت). Therefore, if someone registered “أنترنت”, he/she should be able to enable “إنترنت” or “انترنت”. Additionally, many words have two correct ways to write them, for example, both “آدم” and “أدم” are widely used for the same name Adam. Hence, if someone registered “أدم” he/she should be able to enable “آدم” or “ادم”. This is achieved by making variants allocatable in at least one direction in the LGR.

  • Even though we are constructing an LGR at a language level, because we are addressing international reachability, we need to prevent mixing between code points from different language tables while generating a valid variant. Otherwise, some of permuted variants will be composed of characters (code points) that are not part of a single language or they are not easily available for a single input device (keyboard). Therefore, from a practical and realistic point of view, and to significantly minimize the number of allocatable variants and maximize the number of blocked variants, it blocking these unrealistic variants represents best practice.

  • Based on the principle expressed in the previous item, code points not part of the Arabic language repertoire have been added to variant sets to facilitate international reachability. Many of these out-of-repertoire code points are involved in at least one variant mapping of type “activated”, “optionally-activated”, or “optionally-allocatable. The dispositions related to the extended variants may be optionally modified by a registry by editing the appropriate action elements. See the section on “Customizing Actions for International Reachability” for details of how this feature can be made mandatory or disabled.

Note: Please note that the set of supported languages used for implementing the above concepts (namely international reachability and no mixing between languages) is a first version. The set may be updated in the future by adding new languages whenever they become ready.

The specification of variants in this LGR follows the guidelines in [RFC 8228].

Character Classes

This proposal defines the following named character classes:

  • common-digits — ASCII digits: U+0030 to U+0039
  • arabic-indic-digits — Arabic-Indic digits used with Arabic: U+0660 to U+0669
  • extended arabic-indic-digits — Arabic-Indic digits used with other languages: U+06F0 to U+06F9
  • arabic-language — code points used in pure Arabic language labels
  • pashto-language — code points used in pure Pashto language labels
  • persian-language — code points used in pure Persian language labels
  • urdu-language — code points used in pure Urdu language labels
  • dual-joining — joining-type D
  • right-joining — joining-type R
  • joins-to-the right — union of joining-type R and joining-type D

Whole Label Evaluation (WLE) and Context Rules

Common Rules

By default, the LGR includes the rules and actions to implement the following restrictions mandated by the IDNA protocol. They are marked with ⍟.

  • Hyphen Restrictions — restrictions on the allowable placement of hyphens (no leading/ending hyphen and no hyphen in positions 3 and 4). These restrictions are described in Section 4.2.3.1 of RFC 5891 [150]. They are implemented here as context rule on U+002D (-) HYPHEN-MINUS.
  • Leading Combining Marks — restrictions on the allowable placement of combining marks (no leading combining mark). This rule is described in Section 4.2.3.2 of RFC 5891 [150].

Right-to-Left Rules

  • leading-digit — restrictions on the allowable placement of digits in a right-to-left context (no leading digit), (see section 2.1 of RFC 5893 [160]); implemented here as a context rule on each digit.

Repertoire

Glyph Code point Tags and context Variants
- U+002D HYPHEN-MINUS 0 HYPHEN-MINUS; Tagsc:Zyyy Contextnot hyphen-minus-disallowed
0 U+0030 DIGIT ZERO 0 DIGIT ZERO; TagCommon-digitTagsc:Zyyy Contextnot leading-digit ٠ U+0660 activated Arabic digit variant ۰ U+06F0 optionally-activated Extended digit variant
1 U+0031 DIGIT ONE 0 DIGIT ONE; TagCommon-digitTagsc:Zyyy Contextnot leading-digit ١ U+0661 activated Arabic digit variant ۱ U+06F1 optionally-activated Extended digit variant
2 U+0032 DIGIT TWO 0 DIGIT TWO; TagCommon-digitTagsc:Zyyy Contextnot leading-digit ٢ U+0662 activated Arabic digit variant ۲ U+06F2 optionally-activated Extended digit variant
3 U+0033 DIGIT THREE 0 DIGIT THREE; TagCommon-digitTagsc:Zyyy Contextnot leading-digit ٣ U+0663 activated Arabic digit variant ۳ U+06F3 optionally-activated Extended digit variant
4 U+0034 DIGIT FOUR 0 DIGIT FOUR; TagCommon-digitTagsc:Zyyy Contextnot leading-digit ٤ U+0664 activated Arabic digit variant ۴ U+06F4 optionally-activated Extended digit variant
5 U+0035 DIGIT FIVE 0 DIGIT FIVE; TagCommon-digitTagsc:Zyyy Contextnot leading-digit ٥ U+0665 activated Arabic digit variant ۵ U+06F5 optionally-activated Extended digit variant
6 U+0036 DIGIT SIX 0 DIGIT SIX; TagCommon-digitTagsc:Zyyy Contextnot leading-digit ٦ U+0666 activated Arabic digit variant ۶ U+06F6 optionally-activated Extended digit variant
7 U+0037 DIGIT SEVEN 0 DIGIT SEVEN; TagCommon-digitTagsc:Zyyy Contextnot leading-digit ٧ U+0667 activated Arabic digit variant ۷ U+06F7 optionally-activated Extended digit variant
8 U+0038 DIGIT EIGHT 0 DIGIT EIGHT; TagCommon-digitTagsc:Zyyy Contextnot leading-digit ٨ U+0668 activated Arabic digit variant ۸ U+06F8 optionally-activated Extended digit variant
9 U+0039 DIGIT NINE 0 DIGIT NINE; TagCommon-digitTagsc:Zyyy Contextnot leading-digit ٩ U+0669 activated Arabic digit variant ۹ U+06F9 optionally-activated Extended digit variant
ء U+0621 ARABIC LETTER HAMZA 0 100 Arabic Tagsc:Arab
آ U+0622 ARABIC LETTER ALEF WITH MADDA ABOVE 0 100 Arabic Tagsc:Arab أ U+0623 optionally-allocatable Arabic language variant إ U+0625 optionally-allocatable Arabic language variant ا U+0627 allocatable U+0622 ALEF WITH MADDA ABOVE is simplified to U+0627 ALEF in the Arabic language
أ U+0623 ARABIC LETTER ALEF WITH HAMZA ABOVE 0 100 Arabic Tagsc:Arab آ U+0622 optionally-allocatable Arabic language variant إ U+0625 optionally-allocatable Arabic language variant ا U+0627 allocatable U+0623 ALEF WITH HAMZA ABOVE is simplified to U+0627 ALEF in the Arabic language
ؤ U+0624 ARABIC LETTER WAW WITH HAMZA ABOVE 0 100 Arabic Tagsc:Arab و U+0648 allocatable U+0624 WAV WITH HAMZA ABOVE is simplified to U+0648 WAV in the Arabic language
إ U+0625 ARABIC LETTER ALEF WITH HAMZA BELOW 0 100 Arabic Tagsc:Arab آ U+0622 optionally-allocatable Arabic language variant أ U+0623 optionally-allocatable Arabic language variant ا U+0627 allocatable U+0625 ALEF WITH HAMZA BELOW is simplified to U+0627 ALEF in the Arabic language
ئ U+0626 ARABIC LETTER YEH WITH HAMZA ABOVE 0 100 Arabic Tagsc:Arab ى U+0649 blocked Arabic language variant ي U+064A blocked Arabic language variant ی U+06CC blocked Cross-repertoire variant ۍ U+06CD blocked Cross-repertoire variant ې U+06D0 blocked Cross-repertoire variant ے U+06D2 blocked Cross-repertoire variant
ا U+0627 ARABIC LETTER ALEF 0 100 Arabic Tagsc:Arab آ U+0622 optionally-allocatable Arabic language variant أ U+0623 optionally-allocatable Arabic language variant إ U+0625 optionally-allocatable Arabic language variant
ب U+0628 ARABIC LETTER BEH 0 100 Arabic Tagsc:Arab
ة U+0629 ARABIC LETTER TEH MARBUTA 0 100 Arabic Tagsc:Arab ه U+0647 allocatable In the Arabic language, U+0647 HEH may be substituted for U+0629 TEH MARBUTA. [RFC 6365] ھ U+06BE blocked Cross-repertoire variant ہ U+06C1 blocked Cross-repertoire variant ۃ U+06C3 optionally-activated
Comment The two are visually identical and the same label could be typed using one or the other based on the set language settings and keyboard layout of a user. Labels in the Arabic language using U+0629 TEH MARBUTA in the final and isolated positions will be typed in other languages using U+06C3 TEH MARBUTA GOAL (Urdu, etc.) which is identical in isolated and has a variant glyph or identical glyph form in final position
ت U+062A ARABIC LETTER TEH 0 100 Arabic Tagsc:Arab
ث U+062B ARABIC LETTER THEH 0 100 Arabic Tagsc:Arab
ج U+062C ARABIC LETTER JEEM 0 100 Arabic Tagsc:Arab
ح U+062D ARABIC LETTER HAH 0 100 Arabic Tagsc:Arab
خ U+062E ARABIC LETTER KHAH 0 100 Arabic Tagsc:Arab
د U+062F ARABIC LETTER DAL 0 100 Arabic Tagsc:Arab
ذ U+0630 ARABIC LETTER THAL 0 100 Arabic Tagsc:Arab
ر U+0631 ARABIC LETTER REH 0 100 Arabic Tagsc:Arab
ز U+0632 ARABIC LETTER ZAIN 0 100 Arabic Tagsc:Arab
س U+0633 ARABIC LETTER SEEN 0 100 Arabic Tagsc:Arab
ش U+0634 ARABIC LETTER SHEEN 0 100 Arabic Tagsc:Arab
ص U+0635 ARABIC LETTER SAD 0 100 Arabic Tagsc:Arab
ض U+0636 ARABIC LETTER DAD 0 100 Arabic Tagsc:Arab
ط U+0637 ARABIC LETTER TAH 0 100 Arabic Tagsc:Arab
ظ U+0638 ARABIC LETTER ZAH 0 100 Arabic Tagsc:Arab
ع U+0639 ARABIC LETTER AIN 0 100 Arabic Tagsc:Arab
غ U+063A ARABIC LETTER GHAIN 0 100 Arabic Tagsc:Arab
ف U+0641 ARABIC LETTER FEH 0 100 Arabic Tagsc:Arab
ق U+0642 ARABIC LETTER QAF 0 100 Arabic Tagsc:Arab
ك U+0643 ARABIC LETTER KAF 0 100 Arabic Tagsc:Arab ک U+06A9 optionally-activated
Comment The two have identical shapes in initial and medial positions and are used by different language communities to refer to the same letter
ل U+0644 ARABIC LETTER LAM 0 100 Arabic Tagsc:Arab
م U+0645 ARABIC LETTER MEEM 0 100 Arabic Tagsc:Arab
ن U+0646 ARABIC LETTER NOON 0 100 Arabic Tagsc:Arab ں U+06BA blocked Cross-repertoire variant
ه U+0647 ARABIC LETTER HEH 0 100 Arabic Tagsc:Arab ة U+0629 blocked Arabic language variant ھ U+06BE optionally-activated
Comment Labels in the Arabic language using U+0647 HEH in the initial and medial positions will be visually similar in Urdu language to U+06BE LETTER HEH DOACHASHMEE
ہ U+06C1 optionally-activated
Comment Labels in the Arabic language using U+0647 HEH in the final and isolated positions will be typed in other languages using U+06C1 HEH GOAL (Urdu, Pashto, Saraiki, etc.) which is identical in isolated and has a variant glyph or identical glyph form in final position
ۃ U+06C3 blocked Cross-repertoire variant
و U+0648 ARABIC LETTER WAW 0 100 Arabic Tagsc:Arab ؤ U+0624 blocked Arabic language variant
ى U+0649 ARABIC LETTER ALEF MAKSURA 0 100 Arabic Tagsc:Arab Contextnot initial-or-medial-position ئ U+0626 blocked Arabic language variant ي U+064A blocked Arabic language variant ی U+06CC optionally-activated
Comment The two are visually identical and the same label could be typed using one or the other based on the set language settings and keyboard layout of a user
ۍ U+06CD blocked Cross-repertoire variant ې U+06D0 blocked Cross-repertoire variant ے U+06D2 blocked Cross-repertoire variant
ي U+064A ARABIC LETTER YEH 0 100 Arabic Tagsc:Arab ئ U+0626 blocked Arabic language variant ى U+0649 blocked Arabic language variant ی U+06CC optionally-activated
Comment The two are visually identical and the same label could be typed using one or the other based on the set language settings and keyboard layout of a user
ۍ U+06CD blocked Cross-repertoire variant ې U+06D0 blocked Cross-repertoire variant ے U+06D2 blocked Cross-repertoire variant
٠ U+0660 ARABIC-INDIC DIGIT ZERO 0 100 201 401 600 700 ARABIC-INDIC DIGIT ZERO; TagArabic-indic-digitTagsc:Arab Contextnot leading-digit 0 U+0030 activated ASCII digit variant ۰ U+06F0 optionally-activated Extended digit variant
١ U+0661 ARABIC-INDIC DIGIT ONE 0 100 201 401 600 700 ARABIC-INDIC DIGIT ONE; TagArabic-indic-digitTagsc:Arab Contextnot leading-digit 1 U+0031 activated ASCII digit variant ۱ U+06F1 optionally-activated Extended digit variant
٢ U+0662 ARABIC-INDIC DIGIT TWO 0 100 201 401 600 700 ARABIC-INDIC DIGIT TWO; TagArabic-indic-digitTagsc:Arab Contextnot leading-digit 2 U+0032 activated ASCII digit variant ۲ U+06F2 optionally-activated Extended digit variant
٣ U+0663 ARABIC-INDIC DIGIT THREE 0 100 201 401 600 700 ARABIC-INDIC DIGIT THREE; TagArabic-indic-digitTagsc:Arab Contextnot leading-digit 3 U+0033 activated ASCII digit variant ۳ U+06F3 optionally-activated Extended digit variant
٤ U+0664 ARABIC-INDIC DIGIT FOUR 0 100 201 401 600 700 ARABIC-INDIC DIGIT FOUR; TagArabic-indic-digitTagsc:Arab Contextnot leading-digit 4 U+0034 activated ASCII digit variant ۴ U+06F4 optionally-activated Extended digit variant
٥ U+0665 ARABIC-INDIC DIGIT FIVE 0 100 201 401 600 700 ARABIC-INDIC DIGIT FIVE; TagArabic-indic-digitTagsc:Arab Contextnot leading-digit 5 U+0035 activated ASCII digit variant ۵ U+06F5 optionally-activated Extended digit variant
٦ U+0666 ARABIC-INDIC DIGIT SIX 0 100 201 401 600 700 ARABIC-INDIC DIGIT SIX; TagArabic-indic-digitTagsc:Arab Contextnot leading-digit 6 U+0036 activated ASCII digit variant ۶ U+06F6 optionally-activated Extended digit variant
٧ U+0667 ARABIC-INDIC DIGIT SEVEN 0 100 201 401 600 700 ARABIC-INDIC DIGIT SEVEN; TagArabic-indic-digitTagsc:Arab Contextnot leading-digit 7 U+0037 activated ASCII digit variant ۷ U+06F7 optionally-activated Extended digit variant
٨ U+0668 ARABIC-INDIC DIGIT EIGHT 0 100 201 401 600 700 ARABIC-INDIC DIGIT EIGHT; TagArabic-indic-digitTagsc:Arab Contextnot leading-digit 8 U+0038 activated ASCII digit variant ۸ U+06F8 optionally-activated Extended digit variant
٩ U+0669 ARABIC-INDIC DIGIT NINE 0 100 201 401 600 700 ARABIC-INDIC DIGIT NINE; TagArabic-indic-digitTagsc:Arab Contextnot leading-digit 9 U+0039 activated ASCII digit variant ۹ U+06F9 optionally-activated Extended digit variant
ک U+06A9 ARABIC LETTER KEHEH 0 108 109 112 Pashto, Persian, Urdu; Not available for original label Tagsc:Arab ك U+0643 blocked Cross-repertoire variant
ں U+06BA ARABIC LETTER NOON GHUNNA 0 112 Urdu; Not available for original label Tagsc:Arab ن U+0646 blocked Cross-repertoire variant
ھ U+06BE ARABIC LETTER HEH DOACHASHMEE 0 112 Urdu; Not available for original label Tagsc:Arab ة U+0629 blocked Cross-repertoire variant ه U+0647 blocked Cross-repertoire variant ہ U+06C1 blocked Added for transitivity ۃ U+06C3 blocked Added for transitivity
ہ U+06C1 ARABIC LETTER HEH GOAL 0 112 Urdu; Not available for original label Tagsc:Arab ة U+0629 blocked Cross-repertoire variant ه U+0647 blocked Cross-repertoire variant ھ U+06BE blocked Added for transitivity ۃ U+06C3 blocked Added for transitivity
ۃ U+06C3 ARABIC LETTER TEH MARBUTA GOAL 0 126 Urdu; Not available for original label Tagsc:Arab ة U+0629 blocked Cross-repertoire variant ه U+0647 blocked Cross-repertoire variant ھ U+06BE blocked Added for transitivity ہ U+06C1 blocked Added for transitivity
ی U+06CC ARABIC LETTER FARSI YEH 0 108 109 112 Pashto, Persian, Urdu; Not available for original label Tagsc:Arab ئ U+0626 blocked Arabic language variant ى U+0649 blocked Cross-repertoire variant ي U+064A blocked Cross-repertoire variant ۍ U+06CD blocked Added for transitivity ې U+06D0 blocked Added for transitivity ے U+06D2 blocked Added for transitivity
ۍ U+06CD ARABIC LETTER YEH WITH TAIL 0 108 Pashto; Not available for original label Tagsc:Arab ئ U+0626 blocked Arabic language variant ى U+0649 blocked Cross-repertoire variant ي U+064A blocked Cross-repertoire variant ی U+06CC blocked Cross-repertoire variant ې U+06D0 blocked Added for transitivity ے U+06D2 blocked Added for transitivity
ې U+06D0 ARABIC LETTER E 0 108 Pashto; Not available for original label Tagsc:Arab ئ U+0626 blocked Arabic language variant ى U+0649 blocked Cross-repertoire variant ي U+064A blocked Cross-repertoire variant ی U+06CC blocked Cross-repertoire variant ۍ U+06CD blocked Cross-repertoire variant ے U+06D2 blocked Cross-repertoire variant
ے U+06D2 ARABIC LETTER YEH BARREE 0 112 Urdu; Not available for original label Tagsc:Arab ئ U+0626 blocked Arabic language variant ى U+0649 blocked Cross-repertoire variant ي U+064A blocked Cross-repertoire variant ی U+06CC blocked Cross-repertoire variant ۍ U+06CD blocked Cross-repertoire variant ې U+06D0 blocked Cross-repertoire variant
۰ U+06F0 EXTENDED ARABIC-INDIC DIGIT ZERO 0 109 TagExtended-arabic-indic-digitTagsc:Arab Contextnot leading-digit 0 U+0030 blocked ASCII digit variant ٠ U+0660 blocked Arabic digit variant
۱ U+06F1 EXTENDED ARABIC-INDIC DIGIT ONE 0 109 TagExtended-arabic-indic-digitTagsc:Arab Contextnot leading-digit 1 U+0031 blocked ASCII digit variant ١ U+0661 blocked Arabic digit variant
۲ U+06F2 EXTENDED ARABIC-INDIC DIGIT TWO 0 109 TagExtended-arabic-indic-digitTagsc:Arab Contextnot leading-digit 2 U+0032 blocked ASCII digit variant ٢ U+0662 blocked Arabic digit variant
۳ U+06F3 EXTENDED ARABIC-INDIC DIGIT THREE 0 109 TagExtended-arabic-indic-digitTagsc:Arab Contextnot leading-digit 3 U+0033 blocked ASCII digit variant ٣ U+0663 blocked Arabic digit variant
۴ U+06F4 EXTENDED ARABIC-INDIC DIGIT FOUR 0 109 TagExtended-arabic-indic-digitTagsc:Arab Contextnot leading-digit 4 U+0034 blocked ASCII digit variant ٤ U+0664 blocked Arabic digit variant
۵ U+06F5 EXTENDED ARABIC-INDIC DIGIT FIVE 0 109 TagExtended-arabic-indic-digitTagsc:Arab Contextnot leading-digit 5 U+0035 blocked ASCII digit variant ٥ U+0665 blocked Arabic digit variant
۶ U+06F6 EXTENDED ARABIC-INDIC DIGIT SIX 0 109 TagExtended-arabic-indic-digitTagsc:Arab Contextnot leading-digit 6 U+0036 blocked ASCII digit variant ٦ U+0666 blocked Arabic digit variant
۷ U+06F7 EXTENDED ARABIC-INDIC DIGIT SEVEN 0 109 TagExtended-arabic-indic-digitTagsc:Arab Contextnot leading-digit 7 U+0037 blocked ASCII digit variant ٧ U+0667 blocked Arabic digit variant
۸ U+06F8 EXTENDED ARABIC-INDIC DIGIT EIGHT 0 109 TagExtended-arabic-indic-digitTagsc:Arab Contextnot leading-digit 8 U+0038 blocked ASCII digit variant ٨ U+0668 blocked Arabic digit variant
۹ U+06F9 EXTENDED ARABIC-INDIC DIGIT NINE 0 109 TagExtended-arabic-indic-digitTagsc:Arab Contextnot leading-digit 9 U+0039 blocked ASCII digit variant ٩ U+0669 blocked Arabic digit variant

Identity mappings, where a code point is declared a variant of itself, are required by RFC 7940 and are omitted here. 159 variant records are declared in total.

Variants

Code points grouped into sets of mutual variants. Every member of a set produces the same set of variant labels, so a registration of one has implications for all the others. Sets of a single member are omitted, as are the identity mappings RFC 7940 requires each member to declare.

List
16 variant sets, containing 140 variants.

Character classes

Named sets of code points that the rules below refer to.

NameDefinition
right-joining {jt:R}
dual-joining {jt:D}
joins-to-the-right ({right-joining} ∪ {dual-joining})
arabic-language [U+002D 0030-0039 0621-063A 0641-064A 0660-0669]
urdu-language [U+002D 0030-0039 0621-0622 0626-0628 062A-063A 0641-0642 0644-0646 U+0648 U+0679 U+067E U+0686 U+0688 U+0691 U+0698 U+06A9 U+06AF U+06BA U+06BE U+06C1 U+06C3 U+06CC U+06D2 06F0-06F9]
persian-language [U+002D 0030-0039 0621-0622 0626-0628 062A-063A 0641-0642 0644-0648 U+067E U+0686 U+0698 U+06A9 U+06AF U+06CC 06F0-06F9]
pashto-language [U+002D 0030-0039 0621-0622 U+0624 0626-0628 062A-063A 0641-0642 0644-0648 U+064A U+067C U+067E U+0681 0685-0686 U+0689 U+0693 U+0696 U+0698 U+069A U+06A9 U+06AB U+06BC 06CC-06CD U+06D2 06F0-06F9]
common-digits {tag Common-digit} Digits from the ASCII range MSR default
arabic-indic-digits {tag Arabic-indic-digit} Arabic-Indic digits; MSR default
extended-arabic-indic-digits {tag Extended-arabic-indic-digit} Extended Arabic-Indic digits

Rules

Conditions evaluated against a whole label, or against the context around a single code point. The notation borrows from regular expressions: ^ and $ anchor to the start and end of a label, . matches any code point, (?= ) and (?<= ) look ahead and behind, and * and + repeat. Two symbols are specific to rulesets: ◆ is the code point being tested, and ⟨name⟩ refers to another rule.

leading-combining-mark MSR default 150

RFC 5891 restrictions on placement of combining marks

^ ({gc:Mn} ∪ {gc:Mc})
Structure
  • ^ start
  • ({gc:Mn} ∪ {gc:Mc}) union
    • {gc:Mn} class
    • {gc:Mc} class
Referenced by 1 definition
  • action 3 (invalid)

hyphen-minus-disallowed MSR default 150

RFC 5891 restrictions on placement of U+002D HYPHEN-MINUS

( ( (?<= ^ ) ◆ ) │ ( ◆ (?= $ ) ) │ ( (?<= ^ . . U+002D ) ◆ ) )
Structure
  • ( ( (?<= ^ ) ◆ ) │ ( ◆ (?= $ ) ) │ ( (?<= ^ . . U+002D ) ◆ ) ) choice
    • (?<= ^ ) ◆ rule no leading hyphen
      • (?<= ^ ) look-behind
        • ^ start
      • ◆ anchor
    • ◆ (?= $ ) rule no trailing hyphen
      • ◆ anchor
      • (?= $ ) look-ahead
        • $ end
    • (?<= ^ . . U+002D ) ◆ rule no consecutive hyphens in third and fourth position in the label
      • (?<= ^ . . U+002D ) look-behind
        • ^ start
        • . any
        • . any
        • U+002D char hyphen-minus
      • ◆ anchor
Referenced by 1 definition
  • negative context of U+002D

single-language-label

a label/variant can only be written using characters from one of the 4 languages

^ ( ( {arabic-language}+ ) │ ( {persian-language}+ ) │ ( {urdu-language}+ ) │ ( {pashto-language}+ ) ) $
Structure
  • ^ start
  • ( ( {arabic-language}+ ) │ ( {persian-language}+ ) │ ( {urdu-language}+ ) │ ( {pashto-language}+ ) ) choice
    • {arabic-language}+ rule
      • {arabic-language}+ class
    • {persian-language}+ rule
      • {persian-language}+ class
    • {urdu-language}+ rule
      • {urdu-language}+ class
    • {pashto-language}+ rule
      • {pashto-language}+ class
  • $ end
Referenced by 1 definition
  • action 1 (invalid)

leading-digit MSR default 160

RFC 5893 RTL labels cannot start with a digit

(?<= ^ ) ◆
Structure
  • (?<= ^ ) look-behind
    • ^ start
  • ◆ anchor
Referenced by 30 definitions
  • negative context of U+0030
  • negative context of U+0031
  • negative context of U+0032
  • negative context of U+0033
  • negative context of U+0034
  • negative context of U+0035
  • negative context of U+0036
  • negative context of U+0037
  • negative context of U+0038
  • negative context of U+0039
  • negative context of U+0660
  • negative context of U+0661
  • and 18 more

digit-mixing 160

RFC 5892 and RFC 5893 restrictions on mixing digits; extended to all three sets

( ( {common-digits} .* ( {arabic-indic-digits} │ {extended-arabic-indic-digits} ) ) │ ( {arabic-indic-digits} .* ( {common-digits} │ {extended-arabic-indic-digits} ) ) │ ( {extended-arabic-indic-digits} .* ( {common-digits} │ {arabic-indic-digits} ) ) )
Structure
  • ( ( {common-digits} .* ( {arabic-indic-digits} │ {extended-arabic-indic-digits} ) ) │ ( {arabic-indic-digits} .* ( {common-digits} │ {extended-arabic-indic-digits} ) ) │ ( {extended-arabic-indic-digits} .* ( {common-digits} │ {arabic-indic-digits} ) ) ) choice
    • {common-digits} .* ( {arabic-indic-digits} │ {extended-arabic-indic-digits} ) rule
      • {common-digits} class
      • .* any
      • ( {arabic-indic-digits} │ {extended-arabic-indic-digits} ) choice
        • {arabic-indic-digits} class
        • {extended-arabic-indic-digits} class
    • {arabic-indic-digits} .* ( {common-digits} │ {extended-arabic-indic-digits} ) rule
      • {arabic-indic-digits} class
      • .* any
      • ( {common-digits} │ {extended-arabic-indic-digits} ) choice
        • {common-digits} class
        • {extended-arabic-indic-digits} class
    • {extended-arabic-indic-digits} .* ( {common-digits} │ {arabic-indic-digits} ) rule
      • {extended-arabic-indic-digits} class
      • .* any
      • ( {common-digits} │ {arabic-indic-digits} ) choice
        • {common-digits} class
        • {arabic-indic-digits} class
Referenced by 1 definition
  • action 2 (invalid)

initial-or-medial-position

U+0649 ALEF MAKSURA must not be in initial or medial position

◆ (?= {joins-to-the-right} )
Structure
  • ◆ anchor
  • (?= {joins-to-the-right} ) look-ahead
    • {joins-to-the-right} class
Referenced by 1 definition
  • negative context of U+0649

Actions

Actions are evaluated in the order shown. The first action whose conditions match determines the disposition of the label.

# Disposition Conditions
1 invalid label does not match single-language-label all labels (including variants) must be entirely within one of four languages
2 invalid label matches digit-mixing a label matching the RFC 5892 and RFC 5893 restriction on digit mixing is invalid
3 invalid label matches leading-combining-mark RFC 5892: labels with leading combining marks are invalid MSR default
4 invalid any variant is out-of-repertoire-var disallow code points that are out of repertoire
5 blocked any variant is blocked any variant label containing blocked variants is blocked MSR default
6 blocked any variant is optionally-allocatable optional allocation for certain within-repertoire variants; to make allocation possible, change (blocked) to (allocatable)
7 allocatable any variant is optionally-activated optional delegation for certain cross-repertoire variants; to make delegation mandatory, change (allocatable) to (activated)
8 activated any variant is activated forced delegation for certain in-repertoire variants
9 allocatable all variants are allocatable variant labels with all variants allocatable are allocatable
10 valid any label default catch all (default action)

References

RefSource
0 The Unicode Consortium. The Unicode Standard, Version 1.0, Volume 1 Reading, MA, Addison-Wesley Developers Press, 1991. ISBN 0-201-56788-1 and The Unicode Standard, Version 1.0, Volume 2 Reading, MA, Addison-Wesley Developers Press, 1992. ISBN 0-201-60845-6, as amended by Davis, Mark, Unicode Technical Report #4: The Unicode Standard, Version 1.1 Cupertino, CA, The Unicode Consortium, 1993
Any code point cited was originally encoded in Unicode Version 1.1
100 RFC 5564 Linguistic Guidelines for the Use of the Arabic Language in Internet Domains https://tools.ietf.org/html/rfc5564
108 Omniglot Pashto https://omniglot.com/writing/pashto.htm
109 Omniglot Persian(Farsi) https://omniglot.com/writing/persian.htm
112 Omniglot Urdu https://omniglot.com/writing/urdu.htm
126 Urdu, Teh marbuta goal, Code point UZT 76 of Urdu Zabta Takhti 1.01, the official code page standard for Govt. of Pakistan, approved in 2001; see https://cle.org.pk/Publication/papers/2001/uzt1.01.pdf
150 RFC 5891, Internationalized Domain Names in Applications (IDNA): Protocol https://tools.ietf.org/html/rfc5891
160 RFC 5893, Right-to-Left Scripts for Internationalized Domain Names for Applications (IDNA) https://tools.ietf.org/html/rfc5893
201 Omniglot Arabic https://www.omniglot.com/writing/arabic.htm
401 The Unicode Consortium, Common Locale Data Repository.- CLDR Version 28 (2015-09-16)- Locale Data Summary for Arabic (ar)- https://www.unicode.org/cldr/charts/28/summary/ar.html
Code points cited are from the set of Main Letters
600 Wikipedia Arabic alphabet https://en.wikipedia.org/wiki/Arabic_alphabet accessed 2015-10-31
Code points cited are from the set of Basic letters
700 Saudi Network Information Center (.sa, Saudi Arabia ccTLD) https://www.iana.org/domains/idn-tables/tables/sa_ar_2.0.pdf