tc-lib-unicode-data

Technical overview and integration notes for tc-lib-unicode-data

Updated:

Overview

tc-lib-unicode-data is a data-centric package that ships Unicode lookup tables, mappings, and constants consumed by tc-lib-unicode and related libraries.

Keeping the tables in their own package means a UCD update is a version bump here, with nothing to review in the algorithms that read them.

Repository and API Docs

Project Metadata

ItemValue
Namespace\Com\Tecnick\Unicode\Data
LicenseGNU LGPL v3

Installation

composer require tecnickcom/tc-lib-unicode-data

Where It Fits

Usually indirectly, through tc-lib-unicode. Reach for it directly when you need to inspect a code point’s properties yourself.

Features

Data Coverage

ClassContent
TypeBidi_Class of every code point, with getType() and getBidiClass()
BidiClassBacked enum of the strong, weak and neutral bidirectional classes
ConstantCode points of the bidirectional formatting characters and of some common separators
PatternRegular expressions matching right-to-left and Arabic text
MirrorMirrored form of the characters mirrored in a right-to-left context
BracketPaired brackets, by opening and by closing code point
ArabicJoining types, presentation forms and ligatures, with getJoiningType()
EncodingCharacter code to glyph name maps of 22 font encodings
LatinUnicode to Latin1 character substitutions
IdentityCMap stream for the Identity-H encoding

Integration Role

Generated Unicode Data

Arabic, Bracket, Mirror, Pattern, and Type are generated from the Unicode Character Database, currently UCD 17.0.0. Type::UNICODE_VERSION reports the release the shipped tables were built from.

ClassUCD sourceContent
Typeextracted/DerivedBidiClass.txtBidi_Class of every code point (UAX #9)
Patternextracted/DerivedBidiClass.txtRegular expressions matching right-to-left and Arabic text
MirrorBidiMirroring.txtBidi_Mirroring_Glyph values used by rule L4
BracketBidiBrackets.txtPaired brackets used by rule N0
ArabicArabicShaping.txt, UnicodeData.txtJoining types, presentation forms, and ligatures

Type::UNI lists only the code points whose Bidi_Class is not L; Type::getType() resolves any code point, including the blocks whose unassigned code points default to R, AL, or ET.

\Com\Tecnick\Unicode\Data\Type::getType(0x05D0);          // 'R'
\Com\Tecnick\Unicode\Data\Type::getBidiClass(0x0660);     // BidiClass::AN
\Com\Tecnick\Unicode\Data\Arabic::getJoiningType(0x0628); // 'D'

To rebuild the tables from a newer UCD release:

make gendata UCDVERSION=17.0.0
make qa

Typed Enums

\Com\Tecnick\Unicode\Data\BidiClass is a backed enum of the bidirectional character classes (L, R, AL, EN, ES, ET, AN, CS, NSM, BN, B, S, WS, ON). The backing value is the canonical class abbreviation used in the UCD.

Upgrading from 2.x

Version 3.0 is a breaking release. The relevant API changes:

  • Type::UNI no longer contains the code points whose Bidi_Class is L. Read the class through Type::getType() instead of Type::UNI[$ord] ?? $fallback.
  • Type::getBidiClass() returns BidiClass::L for unlisted code points, and null only for the explicit formatting codes (LRE, LRO, RLE, RLO, PDF, LRI, RLI, FSI, PDI).
  • Pattern::RTL and Pattern::ARABIC are code-point based (u modifier) and require a valid UTF-8 subject.
  • Mirror::UNI follows BidiMirroring.txt: the quotation marks U+2018, U+2019, U+201C, U+201D, U+301D, and U+301E are no longer mirrored.
  • Arabic::SUBSTITUTE rows always hold the four [isolated, final, initial, medial] forms, repeating the isolated and final ones where a letter has no initial or medial form.
  • Arabic::END is derived from the Joining_Type property and now lists every character that does not join to the following one.
  • Arabic::JOINING and Arabic::getJoiningType() expose the Joining_Type property.

Consumers should upgrade tc-lib-unicode and tc-lib-pdf-font at the same time; both read the bidi class through the Type::getType() and Type::getBidiClass() accessors rather than through the raw table.

Integration Notes

Upgrade this package together with tc-lib-unicode and tc-lib-pdf-font, which read the same tables through accessors that changed alongside them.

Read the data through the accessors rather than indexing the constants. Type::UNI holds only the code points whose Bidi_Class is not L, so a direct lookup misses the blocks whose unassigned code points default to R, AL, or ET.

A UCD release adds code points and occasionally reclassifies existing ones. Run the multilingual cases you care about after an update.

Requirements

  • PHP 8.2 or later
  • Composer

Example

<?php

require_once __DIR__ . '/vendor/autoload.php';

echo md5(\Com\Tecnick\Unicode\Data\Identity::CIDHMAP);

Development and Packaging

  • QA and local checks: make deps, make help, make qa
  • Coverage report: make qa-coverage
  • Packaging: make rpm, make deb

Support and Contribution