Module: Vangrail::Confusables

Defined in:
lib/vangrail/confusables.rb,
lib/vangrail/confusables_data.rb

Overview

Folds characters that imitate ASCII back to the ASCII they imitate.

The data is generated from the Unicode confusables file (UTS #39) and lives in confusables_data.rb. This is the policy that uses it, kept apart so regenerating the table cannot overwrite a decision.

The decision worth writing down is which words to fold. Folding everything turns a page of Russian into ASCII noise: "Кластер" becomes "Kлacтep", which is not Russian, not English, and not what anybody wrote. It happens to be harmless here, because a folded variant is only ever shown to a pattern and never to a reader, but it is the wrong thing to be doing and it widens the surface for an accidental match.

What an imitation attack actually looks like is a mixed word: Latin letters with one or two lookalikes dropped in, so that "system" reads as "system" and is not. A word written entirely in Cyrillic is not imitating anything; it is a word in Cyrillic. UTS #39 draws the same line and calls it mixed-script detection.

So fold leaves single-script words alone and folds the mixed ones. That keeps genuine multilingual documentation intact and still catches the attack, which the corpus asserts in both directions.

Constant Summary collapse

WORD =

A run of non-space characters. Folding is decided per word because that is the unit the mixing happens in.

/\S+/
MAP =

Characters that look like ASCII and are not, mapped to what they imitate.

GENERATED FILE. Do not edit by hand; rerun script/generate_confusables.rb when Unicode moves. The policy for using this data lives in confusables.rb, which is hand-written and survives regeneration.

Source: Unicode confusables data (UTS #39) via the unicode-confusable gem 1.13.0, Unicode 17.0.0.

1645 entries, against the twenty-nine a person can be bothered to write out by hand. The difference is Cherokee, Armenian, the mathematical alphabets, the enclosed forms, and every other neighbourhood nobody remembers while writing a table.

{
  "\u00B4" => "'",
  "\u00B8" => ",",
  "\u00C6" => "AE",
  "\u00D7" => "x",
  "\u00E6" => "ae",
  "\u00FE" => "p",
  "\u0131" => "i",
  "\u0132" => "1J",
  "\u0133" => "ij",
  "\u0149" => "'n",
  "\u0152" => "0E",
  "\u0153" => "oe",
  "\u017F" => "f",
  "\u0181" => "'B",
  "\u0184" => "b",
  "\u0187" => "C'",
  "\u018A" => "'D",
  "\u018D" => "g",
  "\u0192" => "f",
  "\u0193" => "G'",
  "\u0196" => "1",
  "\u0198" => "K'",
  "\u01A4" => "'P",
  "\u01A6" => "R",
  "\u01A7" => "2",
  "\u01AC" => "'T",
  "\u01B3" => "'Y",
  "\u01B7" => "3",
  "\u01BC" => "5",
  "\u01BD" => "s",
  "\u01BF" => "p",
  "\u01C0" => "1",
  "\u01C1" => "11",
  "\u01C3" => "!",
  "\u01C7" => "LJ",
  "\u01C8" => "Lj",
  "\u01C9" => "1j",
  "\u01CA" => "NJ",
  "\u01CB" => "Nj",
  "\u01CC" => "nj",
  "\u01F1" => "DZ",
  "\u01F2" => "Dz",
  "\u01F3" => "dz",
  "\u021C" => "3",
  "\u0222" => "8",
  "\u0223" => "8",
  "\u0241" => "?",
  "\u0251" => "a",
  "\u0261" => "g",
  "\u0263" => "y",
  "\u0269" => "i",
  "\u026A" => "i",
  "\u026F" => "w",
  "\u028B" => "u",
  "\u028F" => "y",
  "\u0294" => "?",
  "\u02A3" => "dz",
  "\u02A6" => "ts",
  "\u02AA" => "1s",
  "\u02AB" => "1z",
  "\u02B9" => "'",
  "\u02BA" => "\"",
  "\u02BB" => "'",
  "\u02BC" => "'",
  "\u02BD" => "'",
  "\u02BE" => "'",
  "\u02C2" => "<",
  "\u02C3" => ">",
  "\u02C4" => "^",
  "\u02C6" => "^",
  "\u02C8" => "'",
  "\u02CA" => "'",
  "\u02CB" => "'",
  "\u02D0" => ":",
  "\u02D7" => "-",
  "\u02DB" => "i",
  "\u02DC" => "~",
  "\u02DD" => "\"",
  "\u02EE" => "\"",
  "\u02F4" => "'",
  "\u02F6" => "\"",
  "\u02F8" => ":",
  "\u0374" => "'",
  "\u037A" => "i",
  "\u037E" => ";",
  "\u037F" => "J",
  "\u0384" => "'",
  "\u0391" => "A",
  "\u0392" => "B",
  "\u0395" => "E",
  "\u0396" => "Z",
  "\u0397" => "H",
  "\u0399" => "1",
  "\u039A" => "K",
  "\u039C" => "M",
  "\u039D" => "N",
  "\u039F" => "0",
  "\u03A1" => "P",
  "\u03A4" => "T",
  "\u03A5" => "Y",
  "\u03A7" => "X",
  "\u03B1" => "a",
  "\u03B3" => "y",
  "\u03B9" => "i",
  "\u03BD" => "v",
  "\u03BF" => "o",
  "\u03C1" => "p",
  "\u03C3" => "o",
  "\u03C5" => "u",
  "\u03D2" => "Y",
  "\u03DC" => "F",
  "\u03E8" => "2",
  "\u03EC" => "6",
  "\u03ED" => "o",
  "\u03F1" => "p",
  "\u03F2" => "c",
  "\u03F3" => "j",
  "\u03F8" => "p",
  "\u03F9" => "C",
  "\u03FA" => "M",
  "\u0405" => "S",
  "\u0406" => "1",
  "\u0408" => "J",
  "\u0410" => "A",
  "\u0412" => "B",
  "\u0415" => "E",
  "\u0417" => "3",
  "\u041A" => "K",
  "\u041C" => "M",
  "\u041D" => "H",
  "\u041E" => "0",
  "\u0420" => "P",
  "\u0421" => "C",
  "\u0422" => "T",
  "\u0423" => "Y",
  "\u0425" => "X",
  "\u042B" => "b1",
  "\u042C" => "b",
  "\u042E" => "10",
  "\u0430" => "a",
  "\u0431" => "6",
  "\u0433" => "r",
  "\u0435" => "e",
  "\u043E" => "o",
  "\u0440" => "p",
  "\u0441" => "c",
  "\u0443" => "y",
  "\u0445" => "x",
  "\u0448" => "w",
  "\u0455" => "s",
  "\u0456" => "i",
  "\u0458" => "j",
  "\u0461" => "w",
  "\u0474" => "V",
  "\u0475" => "v",
  "\u0491" => "r'",
  "\u04AE" => "Y",
  "\u04AF" => "y",
  "\u04BB" => "h",
  "\u04BD" => "e",
  "\u04C0" => "1",
  "\u04CF" => "1",
  "\u04D4" => "AE",
  "\u04D5" => "ae",
  "\u04E0" => "3",
  "\u0501" => "d",
  "\u050C" => "G",
  "\u051B" => "q",
  "\u051C" => "W",
  "\u051D" => "w",
  "\u054D" => "U",
  "\u054F" => "S",
  "\u0555" => "0",
  "\u055A" => "'",
  "\u055D" => "'",
  "\u0561" => "w",
  "\u0563" => "q",
  "\u0566" => "q",
  "\u0570" => "h",
  "\u0578" => "n",
  "\u057C" => "n",
  "\u057D" => "u",
  "\u0581" => "g",
  "\u0582" => "i",
  "\u0584" => "f",
  "\u0585" => "o",
  "\u0589" => ":",
  "\u05C0" => "1",
  "\u05C3" => ":",
  "\u05D5" => "1",
  "\u05D8" => "v",
  "\u05D9" => "'",
  "\u05DF" => "1",
  "\u05E1" => "o",
  "\u05F0" => "11",
  "\u05F1" => "1'",
  "\u05F2" => "\"",
  "\u05F3" => "'",
  "\u05F4" => "\"",
  "\u060D" => ",",
  "\u0627" => "1",
  "\u0647" => "o",
  "\u0660" => ".",
  "\u0661" => "1",
  "\u0665" => "o",
  "\u0667" => "V",
  "\u066A" => "%",
  "\u066B" => ",",
  "\u066D" => "*",
  "\u06BE" => "o",
  "\u06C1" => "o",
  "\u06D4" => "-",
  "\u06D5" => "o",
  "\u06F0" => ".",
  "\u06F1" => "1",
  "\u06F5" => "o",
  "\u06F7" => "V",
  "\u0701" => ".",
  "\u0702" => ".",
  "\u0703" => ":",
  "\u0704" => ":",
  "\u07C0" => "0",
  "\u07CA" => "1",
  "\u07F4" => "'",
  "\u07F5" => "'",
  "\u07FA" => "_",
  "\u0903" => ":",
  "\u0966" => "o",
  "\u0969" => "3",
  "\u097D" => "?",
  "\u09E6" => "o",
  "\u09EA" => "8",
  "\u09ED" => "9",
  "\u0A66" => "o",
  "\u0A67" => "9",
  "\u0A6A" => "8",
  "\u0A83" => ":",
  "\u0AE6" => "o",
  "\u0AE9" => "3",
  "\u0B03" => "8",
  "\u0B20" => "0",
  "\u0B66" => "o",
  "\u0B68" => "9",
  "\u0BE6" => "o",
  "\u0C02" => "o",
  "\u0C66" => "o",
  "\u0C82" => "o",
  "\u0CE6" => "0",
  "\u0D02" => "o",
  "\u0D1F" => "s",
  "\u0D20" => "o",
  "\u0D66" => "o",
  "\u0D6D" => "9",
  "\u0D82" => "o",
  "\u0E50" => "o",
  "\u0ED0" => "o",
  "\u1004" => "c",
  "\u101D" => "o",
  "\u1040" => "o",
  "\u105A" => "c",
  "\u10E7" => "y",
  "\u10FF" => "o",
  "\u1200" => "U",
  "\u12D0" => "0",
  "\u13A0" => "D",
  "\u13A1" => "R",
  "\u13A2" => "T",
  "\u13A4" => "0'",
  "\u13A5" => "i",
  "\u13A9" => "Y",
  "\u13AA" => "A",
  "\u13AB" => "J",
  "\u13AC" => "E",
  "\u13AE" => "?",
  "\u13B3" => "W",
  "\u13B7" => "M",
  "\u13BB" => "H",
  "\u13BD" => "Y",
  "\u13C0" => "G",
  "\u13C2" => "h",
  "\u13C3" => "Z",
  "\u13CE" => "4",
  "\u13CF" => "b",
  "\u13D2" => "R",
  "\u13D4" => "W",
  "\u13D5" => "S",
  "\u13D9" => "V",
  "\u13DA" => "S",
  "\u13DE" => "L",
  "\u13DF" => "C",
  "\u13E2" => "P",
  "\u13E6" => "K",
  "\u13E7" => "d",
  "\u13EE" => "6",
  "\u13F3" => "G",
  "\u13F4" => "B",
  "\u1400" => "=",
  "\u142F" => "V",
  "\u1433" => ">",
  "\u1438" => "<",
  "\u144A" => "'",
  "\u144C" => "U",
  "\u1467" => "U'",
  "\u146D" => "P",
  "\u146F" => "d",
  "\u1472" => "b",
  "\u1486" => "P'",
  "\u1487" => "d'",
  "\u1488" => "b'",
  "\u148D" => "J",
  "\u14AA" => "L",
  "\u14BF" => "2",
  "\u1541" => "x",
  "\u157C" => "H",
  "\u157D" => "x",
  "\u1587" => "R",
  "\u15AF" => "b",
  "\u15B4" => "F",
  "\u15C5" => "A",
  "\u15DE" => "D",
  "\u15EA" => "D",
  "\u15F0" => "M",
  "\u15F7" => "B",
  "\u166D" => "X",
  "\u166E" => "x",
  "\u16B2" => "<",
  "\u16B7" => "X",
  "\u16C1" => "1",
  "\u16CC" => "'",
  "\u16D5" => "K",
  "\u16D6" => "M",
  "\u16EC" => ":",
  "\u16ED" => "+",
  "\u1735" => "/",
  "\u17E0" => "o",
  "\u1803" => ":",
  "\u1809" => ":",
  "\u1CD3" => "\"",
  "\u1D04" => "c",
  "\u1D0F" => "o",
  "\u1D11" => "o",
  "\u1D1C" => "u",
  "\u1D20" => "v",
  "\u1D21" => "w",
  "\u1D22" => "z",
  "\u1D26" => "r",
  "\u1D6B" => "ue",
  "\u1D83" => "g",
  "\u1D8C" => "y",
  "\u1E9D" => "f",
  "\u1EFF" => "y",
  "\u1FBD" => "'",
  "\u1FBE" => "i",
  "\u1FBF" => "'",
  "\u1FC0" => "~",
  "\u1FEF" => "'",
  "\u1FFD" => "'",
  "\u1FFE" => "'",
  "\u2010" => "-",
  "\u2011" => "-",
  "\u2012" => "-",
  "\u2013" => "-",
  "\u2016" => "11",
  "\u2018" => "'",
  "\u2019" => "'",
  "\u201A" => ",",
  "\u201B" => "'",
  "\u201C" => "\"",
  "\u201D" => "\"",
  "\u201F" => "\"",
  "\u2024" => ".",
  "\u2025" => "..",
  "\u2026" => "...",
  "\u2032" => "'",
  "\u2033" => "\"",
  "\u2034" => "'''",
  "\u2035" => "'",
  "\u2036" => "\"",
  "\u2037" => "'''",
  "\u2039" => "<",
  "\u203A" => ">",
  "\u203C" => "!!",
  "\u2041" => "/",
  "\u2043" => "-",
  "\u2044" => "/",
  "\u2047" => "??",
  "\u2048" => "?!",
  "\u2049" => "!?",
  "\u204E" => "*",
  "\u2052" => "%",
  "\u2053" => "~",
  "\u2057" => "''''",
  "\u205A" => ":",
  "\u20A8" => "Rs",
  "\u20B6" => "1t",
  "\u2100" => "a/c",
  "\u2101" => "a/s",
  "\u2102" => "C",
  "\u2105" => "c/o",
  "\u2106" => "c/u",
  "\u210A" => "g",
  "\u210B" => "H",
  "\u210C" => "H",
  "\u210D" => "H",
  "\u210E" => "h",
  "\u2110" => "1",
  "\u2111" => "1",
  "\u2112" => "L",
  "\u2113" => "1",
  "\u2115" => "N",
  "\u2116" => "No",
  "\u2119" => "P",
  "\u211A" => "Q",
  "\u211B" => "R",
  "\u211C" => "R",
  "\u211D" => "R",
  "\u2121" => "TEL",
  "\u2124" => "Z",
  "\u2128" => "Z",
  "\u212A" => "K",
  "\u212C" => "B",
  "\u212D" => "C",
  "\u212E" => "e",
  "\u212F" => "e",
  "\u2130" => "E",
  "\u2131" => "F",
  "\u2133" => "M",
  "\u2134" => "o",
  "\u2139" => "i",
  "\u213B" => "FAX",
  "\u213D" => "y",
  "\u2145" => "D",
  "\u2146" => "d",
  "\u2147" => "e",
  "\u2148" => "i",
  "\u2149" => "j",
  "\u2160" => "1",
  "\u2161" => "11",
  "\u2162" => "111",
  "\u2163" => "1V",
  "\u2164" => "V",
  "\u2165" => "V1",
  "\u2166" => "V11",
  "\u2167" => "V111",
  "\u2168" => "1X",
  "\u2169" => "X",
  "\u216A" => "X1",
  "\u216B" => "X11",
  "\u216C" => "L",
  "\u216D" => "C",
  "\u216E" => "D",
  "\u216F" => "M",
  "\u2170" => "i",
  "\u2171" => "ii",
  "\u2172" => "iii",
  "\u2173" => "iv",
  "\u2174" => "v",
  "\u2175" => "vi",
  "\u2176" => "vii",
  "\u2177" => "viii",
  "\u2178" => "ix",
  "\u2179" => "x",
  "\u217A" => "xi",
  "\u217B" => "xii",
  "\u217C" => "1",
  "\u217D" => "c",
  "\u217E" => "d",
  "\u217F" => "m",
  "\u2212" => "-",
  "\u2215" => "/",
  "\u2216" => "\\",
  "\u2217" => "*",
  "\u221E" => "oo",
  "\u2223" => "1",
  "\u2225" => "11",
  "\u2228" => "v",
  "\u222A" => "U",
  "\u2236" => ":",
  "\u223C" => "~",
  "\u226A" => "<<",
  "\u226B" => ">>",
  "\u22A4" => "T",
  "\u22C1" => "v",
  "\u22C3" => "U",
  "\u22D8" => "<<<",
  "\u22D9" => ">>>",
  "\u22FF" => "E",
  "\u2373" => "i",
  "\u2374" => "p",
  "\u237A" => "a",
  "\u23FD" => "1",
  "\u244A" => "\\\\",
  "\u2474" => "(1)",
  "\u2475" => "(2)",
  "\u2476" => "(3)",
  "\u2477" => "(4)",
  "\u2478" => "(5)",
  "\u2479" => "(6)",
  "\u247A" => "(7)",
  "\u247B" => "(8)",
  "\u247C" => "(9)",
  "\u247D" => "(10)",
  "\u247E" => "(11)",
  "\u247F" => "(12)",
  "\u2480" => "(13)",
  "\u2481" => "(14)",
  "\u2482" => "(15)",
  "\u2483" => "(16)",
  "\u2484" => "(17)",
  "\u2485" => "(18)",
  "\u2486" => "(19)",
  "\u2487" => "(20)",
  "\u2488" => "1.",
  "\u2489" => "2.",
  "\u248A" => "3.",
  "\u248B" => "4.",
  "\u248C" => "5.",
  "\u248D" => "6.",
  "\u248E" => "7.",
  "\u248F" => "8.",
  "\u2490" => "9.",
  "\u2491" => "10.",
  "\u2492" => "11.",
  "\u2493" => "12.",
  "\u2494" => "13.",
  "\u2495" => "14.",
  "\u2496" => "15.",
  "\u2497" => "16.",
  "\u2498" => "17.",
  "\u2499" => "18.",
  "\u249A" => "19.",
  "\u249B" => "20.",
  "\u249C" => "(a)",
  "\u249D" => "(b)",
  "\u249E" => "(c)",
  "\u249F" => "(d)",
  "\u24A0" => "(e)",
  "\u24A1" => "(f)",
  "\u24A2" => "(g)",
  "\u24A3" => "(h)",
  "\u24A4" => "(i)",
  "\u24A5" => "(j)",
  "\u24A6" => "(k)",
  "\u24A7" => "(1)",
  "\u24A8" => "(rn)",
  "\u24A9" => "(n)",
  "\u24AA" => "(o)",
  "\u24AB" => "(p)",
  "\u24AC" => "(q)",
  "\u24AD" => "(r)",
  "\u24AE" => "(s)",
  "\u24AF" => "(t)",
  "\u24B0" => "(u)",
  "\u24B1" => "(v)",
  "\u24B2" => "(w)",
  "\u24B3" => "(x)",
  "\u24B4" => "(y)",
  "\u24B5" => "(z)",
  "\u2571" => "/",
  "\u2573" => "X",
  "\u2768" => "(",
  "\u2769" => ")",
  "\u276E" => "<",
  "\u276F" => ">",
  "\u2772" => "(",
  "\u2773" => ")",
  "\u2774" => "{",
  "\u2775" => "}",
  "\u2795" => "+",
  "\u2796" => "-",
  "\u27CB" => "/",
  "\u27CD" => "\\",
  "\u27D9" => "T",
  "\u292B" => "x",
  "\u292C" => "x",
  "\u29F5" => "\\",
  "\u29F8" => "/",
  "\u29F9" => "\\",
  "\u2A20" => ">>",
  "\u2A2F" => "x",
  "\u2A74" => "::=",
  "\u2A75" => "==",
  "\u2A76" => "===",
  "\u2AA5" => "><",
  "\u2AFB" => "///",
  "\u2AFD" => "//",
  "\u2C82" => "B",
  "\u2C85" => "r",
  "\u2C8E" => "H",
  "\u2C92" => "1",
  "\u2C93" => "i",
  "\u2C94" => "K",
  "\u2C98" => "M",
  "\u2C9A" => "N",
  "\u2C9C" => "3",
  "\u2C9E" => "0",
  "\u2C9F" => "o",
  "\u2CA2" => "P",
  "\u2CA3" => "p",
  "\u2CA4" => "C",
  "\u2CA5" => "c",
  "\u2CA6" => "T",
  "\u2CA8" => "Y",
  "\u2CA9" => "y",
  "\u2CAC" => "X",
  "\u2CBA" => "-",
  "\u2CBB" => "-",
  "\u2CBD" => "w",
  "\u2CC4" => "3",
  "\u2CC6" => "/",
  "\u2CC7" => "/",
  "\u2CCA" => "9",
  "\u2CCB" => "9",
  "\u2CCC" => "3",
  "\u2CCE" => "P",
  "\u2CCF" => "p",
  "\u2CD0" => "L",
  "\u2CD2" => "6",
  "\u2CD3" => "6",
  "\u2CDC" => "6",
  "\u2CF9" => "\\\\",
  "\u2D38" => "V",
  "\u2D39" => "E",
  "\u2D4F" => "1",
  "\u2D51" => "!",
  "\u2D54" => "0",
  "\u2D55" => "Q",
  "\u2D5D" => "X",
  "\u2E28" => "((",
  "\u2E29" => "))",
  "\u2E40" => "=",
  "\u2F02" => "\\",
  "\u2F03" => "/",
  "\uA4D0" => "B",
  "\uA4D1" => "P",
  "\uA4D2" => "d",
  "\uA4D3" => "D",
  "\uA4D4" => "T",
  "\uA4D6" => "G",
  "\uA4D7" => "K",
  "\uA4D9" => "J",
  "\uA4DA" => "C",
  "\uA4DC" => "Z",
  "\uA4DD" => "F",
  "\uA4DF" => "M",
  "\uA4E0" => "N",
  "\uA4E1" => "L",
  "\uA4E2" => "S",
  "\uA4E3" => "R",
  "\uA4E6" => "V",
  "\uA4E7" => "H",
  "\uA4EA" => "W",
  "\uA4EB" => "X",
  "\uA4EC" => "Y",
  "\uA4EE" => "A",
  "\uA4F0" => "E",
  "\uA4F2" => "1",
  "\uA4F3" => "0",
  "\uA4F4" => "U",
  "\uA4F8" => ".",
  "\uA4F9" => ",",
  "\uA4FA" => "..",
  "\uA4FB" => ".,",
  "\uA4FD" => ":",
  "\uA4FE" => "-.",
  "\uA4FF" => "=",
  "\uA60E" => ".",
  "\uA644" => "2",
  "\uA647" => "i",
  "\uA698" => "00",
  "\uA699" => "oo",
  "\uA6DF" => "V",
  "\uA6EB" => "?",
  "\uA6EF" => "2",
  "\uA728" => "T3",
  "\uA731" => "s",
  "\uA732" => "AA",
  "\uA733" => "aa",
  "\uA734" => "A0",
  "\uA735" => "ao",
  "\uA736" => "AU",
  "\uA737" => "au",
  "\uA738" => "AV",
  "\uA739" => "av",
  "\uA73A" => "AV",
  "\uA73B" => "av",
  "\uA73C" => "AY",
  "\uA73D" => "ay",
  "\uA74E" => "00",
  "\uA74F" => "oo",
  "\uA75A" => "2",
  "\uA76A" => "3",
  "\uA76E" => "9",
  "\uA777" => "tf",
  "\uA778" => "&",
  "\uA789" => ":",
  "\uA78C" => "'",
  "\uA798" => "F",
  "\uA799" => "f",
  "\uA79F" => "u",
  "\uA7AB" => "3",
  "\uA7B2" => "J",
  "\uA7B3" => "X",
  "\uA7B4" => "B",
  "\uAB32" => "e",
  "\uAB35" => "f",
  "\uAB3D" => "o",
  "\uAB47" => "r",
  "\uAB48" => "r",
  "\uAB4E" => "u",
  "\uAB52" => "u",
  "\uAB5A" => "y",
  "\uAB63" => "uo",
  "\uAB75" => "i",
  "\uAB81" => "r",
  "\uAB83" => "w",
  "\uAB93" => "z",
  "\uABA9" => "v",
  "\uABAA" => "s",
  "\uABAF" => "c",
  "\uFB00" => "ff",
  "\uFB01" => "fi",
  "\uFB02" => "f1",
  "\uFB03" => "ffi",
  "\uFB04" => "ff1",
  "\uFB06" => "st",
  "\uFBA6" => "o",
  "\uFBA7" => "o",
  "\uFBA8" => "o",
  "\uFBA9" => "o",
  "\uFBAA" => "o",
  "\uFBAB" => "o",
  "\uFBAC" => "o",
  "\uFBAD" => "o",
  "\uFD3E" => "(",
  "\uFD3F" => ")",
  "\uFE30" => ":",
  "\uFE4D" => "_",
  "\uFE4E" => "_",
  "\uFE4F" => "_",
  "\uFE58" => "-",
  "\uFE68" => "\\",
  "\uFE8D" => "1",
  "\uFE8E" => "1",
  "\uFEE9" => "o",
  "\uFEEA" => "o",
  "\uFEEB" => "o",
  "\uFEEC" => "o",
  "\uFF01" => "!",
  "\uFF02" => "\"",
  "\uFF07" => "'",
  "\uFF1A" => ":",
  "\uFF21" => "A",
  "\uFF22" => "B",
  "\uFF23" => "C",
  "\uFF25" => "E",
  "\uFF28" => "H",
  "\uFF29" => "1",
  "\uFF2A" => "J",
  "\uFF2B" => "K",
  "\uFF2D" => "M",
  "\uFF2E" => "N",
  "\uFF2F" => "0",
  "\uFF30" => "P",
  "\uFF33" => "S",
  "\uFF34" => "T",
  "\uFF38" => "X",
  "\uFF39" => "Y",
  "\uFF3A" => "Z",
  "\uFF3B" => "(",
  "\uFF3C" => "\\",
  "\uFF3D" => ")",
  "\uFF40" => "'",
  "\uFF41" => "a",
  "\uFF43" => "c",
  "\uFF45" => "e",
  "\uFF47" => "g",
  "\uFF48" => "h",
  "\uFF49" => "i",
  "\uFF4A" => "j",
  "\uFF4C" => "1",
  "\uFF4F" => "o",
  "\uFF50" => "p",
  "\uFF53" => "s",
  "\uFF56" => "v",
  "\uFF58" => "x",
  "\uFF59" => "y",
  "\uFFE8" => "1",
  "\u{1D400}" => "A",
  "\u{1D401}" => "B",
  "\u{1D402}" => "C",
  "\u{1D403}" => "D",
  "\u{1D404}" => "E",
  "\u{1D405}" => "F",
  "\u{1D406}" => "G",
  "\u{1D407}" => "H",
  "\u{1D408}" => "1",
  "\u{1D409}" => "J",
  "\u{1D40A}" => "K",
  "\u{1D40B}" => "L",
  "\u{1D40C}" => "M",
  "\u{1D40D}" => "N",
  "\u{1D40E}" => "0",
  "\u{1D40F}" => "P",
  "\u{1D410}" => "Q",
  "\u{1D411}" => "R",
  "\u{1D412}" => "S",
  "\u{1D413}" => "T",
  "\u{1D414}" => "U",
  "\u{1D415}" => "V",
  "\u{1D416}" => "W",
  "\u{1D417}" => "X",
  "\u{1D418}" => "Y",
  "\u{1D419}" => "Z",
  "\u{1D41A}" => "a",
  "\u{1D41B}" => "b",
  "\u{1D41C}" => "c",
  "\u{1D41D}" => "d",
  "\u{1D41E}" => "e",
  "\u{1D41F}" => "f",
  "\u{1D420}" => "g",
  "\u{1D421}" => "h",
  "\u{1D422}" => "i",
  "\u{1D423}" => "j",
  "\u{1D424}" => "k",
  "\u{1D425}" => "1",
  "\u{1D426}" => "m",
  "\u{1D427}" => "n",
  "\u{1D428}" => "o",
  "\u{1D429}" => "p",
  "\u{1D42A}" => "q",
  "\u{1D42B}" => "r",
  "\u{1D42C}" => "s",
  "\u{1D42D}" => "t",
  "\u{1D42E}" => "u",
  "\u{1D42F}" => "v",
  "\u{1D430}" => "w",
  "\u{1D431}" => "x",
  "\u{1D432}" => "y",
  "\u{1D433}" => "z",
  "\u{1D434}" => "A",
  "\u{1D435}" => "B",
  "\u{1D436}" => "C",
  "\u{1D437}" => "D",
  "\u{1D438}" => "E",
  "\u{1D439}" => "F",
  "\u{1D43A}" => "G",
  "\u{1D43B}" => "H",
  "\u{1D43C}" => "1",
  "\u{1D43D}" => "J",
  "\u{1D43E}" => "K",
  "\u{1D43F}" => "L",
  "\u{1D440}" => "M",
  "\u{1D441}" => "N",
  "\u{1D442}" => "0",
  "\u{1D443}" => "P",
  "\u{1D444}" => "Q",
  "\u{1D445}" => "R",
  "\u{1D446}" => "S",
  "\u{1D447}" => "T",
  "\u{1D448}" => "U",
  "\u{1D449}" => "V",
  "\u{1D44A}" => "W",
  "\u{1D44B}" => "X",
  "\u{1D44C}" => "Y",
  "\u{1D44D}" => "Z",
  "\u{1D44E}" => "a",
  "\u{1D44F}" => "b",
  "\u{1D450}" => "c",
  "\u{1D451}" => "d",
  "\u{1D452}" => "e",
  "\u{1D453}" => "f",
  "\u{1D454}" => "g",
  "\u{1D456}" => "i",
  "\u{1D457}" => "j",
  "\u{1D458}" => "k",
  "\u{1D459}" => "1",
  "\u{1D45A}" => "m",
  "\u{1D45B}" => "n",
  "\u{1D45C}" => "o",
  "\u{1D45D}" => "p",
  "\u{1D45E}" => "q",
  "\u{1D45F}" => "r",
  "\u{1D460}" => "s",
  "\u{1D461}" => "t",
  "\u{1D462}" => "u",
  "\u{1D463}" => "v",
  "\u{1D464}" => "w",
  "\u{1D465}" => "x",
  "\u{1D466}" => "y",
  "\u{1D467}" => "z",
  "\u{1D468}" => "A",
  "\u{1D469}" => "B",
  "\u{1D46A}" => "C",
  "\u{1D46B}" => "D",
  "\u{1D46C}" => "E",
  "\u{1D46D}" => "F",
  "\u{1D46E}" => "G",
  "\u{1D46F}" => "H",
  "\u{1D470}" => "1",
  "\u{1D471}" => "J",
  "\u{1D472}" => "K",
  "\u{1D473}" => "L",
  "\u{1D474}" => "M",
  "\u{1D475}" => "N",
  "\u{1D476}" => "0",
  "\u{1D477}" => "P",
  "\u{1D478}" => "Q",
  "\u{1D479}" => "R",
  "\u{1D47A}" => "S",
  "\u{1D47B}" => "T",
  "\u{1D47C}" => "U",
  "\u{1D47D}" => "V",
  "\u{1D47E}" => "W",
  "\u{1D47F}" => "X",
  "\u{1D480}" => "Y",
  "\u{1D481}" => "Z",
  "\u{1D482}" => "a",
  "\u{1D483}" => "b",
  "\u{1D484}" => "c",
  "\u{1D485}" => "d",
  "\u{1D486}" => "e",
  "\u{1D487}" => "f",
  "\u{1D488}" => "g",
  "\u{1D489}" => "h",
  "\u{1D48A}" => "i",
  "\u{1D48B}" => "j",
  "\u{1D48C}" => "k",
  "\u{1D48D}" => "1",
  "\u{1D48E}" => "m",
  "\u{1D48F}" => "n",
  "\u{1D490}" => "o",
  "\u{1D491}" => "p",
  "\u{1D492}" => "q",
  "\u{1D493}" => "r",
  "\u{1D494}" => "s",
  "\u{1D495}" => "t",
  "\u{1D496}" => "u",
  "\u{1D497}" => "v",
  "\u{1D498}" => "w",
  "\u{1D499}" => "x",
  "\u{1D49A}" => "y",
  "\u{1D49B}" => "z",
  "\u{1D49C}" => "A",
  "\u{1D49E}" => "C",
  "\u{1D49F}" => "D",
  "\u{1D4A2}" => "G",
  "\u{1D4A5}" => "J",
  "\u{1D4A6}" => "K",
  "\u{1D4A9}" => "N",
  "\u{1D4AA}" => "0",
  "\u{1D4AB}" => "P",
  "\u{1D4AC}" => "Q",
  "\u{1D4AE}" => "S",
  "\u{1D4AF}" => "T",
  "\u{1D4B0}" => "U",
  "\u{1D4B1}" => "V",
  "\u{1D4B2}" => "W",
  "\u{1D4B3}" => "X",
  "\u{1D4B4}" => "Y",
  "\u{1D4B5}" => "Z",
  "\u{1D4B6}" => "a",
  "\u{1D4B7}" => "b",
  "\u{1D4B8}" => "c",
  "\u{1D4B9}" => "d",
  "\u{1D4BB}" => "f",
  "\u{1D4BD}" => "h",
  "\u{1D4BE}" => "i",
  "\u{1D4BF}" => "j",
  "\u{1D4C0}" => "k",
  "\u{1D4C1}" => "1",
  "\u{1D4C2}" => "m",
  "\u{1D4C3}" => "n",
  "\u{1D4C5}" => "p",
  "\u{1D4C6}" => "q",
  "\u{1D4C7}" => "r",
  "\u{1D4C8}" => "s",
  "\u{1D4C9}" => "t",
  "\u{1D4CA}" => "u",
  "\u{1D4CB}" => "v",
  "\u{1D4CC}" => "w",
  "\u{1D4CD}" => "x",
  "\u{1D4CE}" => "y",
  "\u{1D4CF}" => "z",
  "\u{1D4D0}" => "A",
  "\u{1D4D1}" => "B",
  "\u{1D4D2}" => "C",
  "\u{1D4D3}" => "D",
  "\u{1D4D4}" => "E",
  "\u{1D4D5}" => "F",
  "\u{1D4D6}" => "G",
  "\u{1D4D7}" => "H",
  "\u{1D4D8}" => "1",
  "\u{1D4D9}" => "J",
  "\u{1D4DA}" => "K",
  "\u{1D4DB}" => "L",
  "\u{1D4DC}" => "M",
  "\u{1D4DD}" => "N",
  "\u{1D4DE}" => "0",
  "\u{1D4DF}" => "P",
  "\u{1D4E0}" => "Q",
  "\u{1D4E1}" => "R",
  "\u{1D4E2}" => "S",
  "\u{1D4E3}" => "T",
  "\u{1D4E4}" => "U",
  "\u{1D4E5}" => "V",
  "\u{1D4E6}" => "W",
  "\u{1D4E7}" => "X",
  "\u{1D4E8}" => "Y",
  "\u{1D4E9}" => "Z",
  "\u{1D4EA}" => "a",
  "\u{1D4EB}" => "b",
  "\u{1D4EC}" => "c",
  "\u{1D4ED}" => "d",
  "\u{1D4EE}" => "e",
  "\u{1D4EF}" => "f",
  "\u{1D4F0}" => "g",
  "\u{1D4F1}" => "h",
  "\u{1D4F2}" => "i",
  "\u{1D4F3}" => "j",
  "\u{1D4F4}" => "k",
  "\u{1D4F5}" => "1",
  "\u{1D4F6}" => "m",
  "\u{1D4F7}" => "n",
  "\u{1D4F8}" => "o",
  "\u{1D4F9}" => "p",
  "\u{1D4FA}" => "q",
  "\u{1D4FB}" => "r",
  "\u{1D4FC}" => "s",
  "\u{1D4FD}" => "t",
  "\u{1D4FE}" => "u",
  "\u{1D4FF}" => "v",
  "\u{1D500}" => "w",
  "\u{1D501}" => "x",
  "\u{1D502}" => "y",
  "\u{1D503}" => "z",
  "\u{1D504}" => "A",
  "\u{1D505}" => "B",
  "\u{1D507}" => "D",
  "\u{1D508}" => "E",
  "\u{1D509}" => "F",
  "\u{1D50A}" => "G",
  "\u{1D50D}" => "J",
  "\u{1D50E}" => "K",
  "\u{1D50F}" => "L",
  "\u{1D510}" => "M",
  "\u{1D511}" => "N",
  "\u{1D512}" => "0",
  "\u{1D513}" => "P",
  "\u{1D514}" => "Q",
  "\u{1D516}" => "S",
  "\u{1D517}" => "T",
  "\u{1D518}" => "U",
  "\u{1D519}" => "V",
  "\u{1D51A}" => "W",
  "\u{1D51B}" => "X",
  "\u{1D51C}" => "Y",
  "\u{1D51E}" => "a",
  "\u{1D51F}" => "b",
  "\u{1D520}" => "c",
  "\u{1D521}" => "d",
  "\u{1D522}" => "e",
  "\u{1D523}" => "f",
  "\u{1D524}" => "g",
  "\u{1D525}" => "h",
  "\u{1D526}" => "i",
  "\u{1D527}" => "j",
  "\u{1D528}" => "k",
  "\u{1D529}" => "1",
  "\u{1D52A}" => "m",
  "\u{1D52B}" => "n",
  "\u{1D52C}" => "o",
  "\u{1D52D}" => "p",
  "\u{1D52E}" => "q",
  "\u{1D52F}" => "r",
  "\u{1D530}" => "s",
  "\u{1D531}" => "t",
  "\u{1D532}" => "u",
  "\u{1D533}" => "v",
  "\u{1D534}" => "w",
  "\u{1D535}" => "x",
  "\u{1D536}" => "y",
  "\u{1D537}" => "z",
  "\u{1D538}" => "A",
  "\u{1D539}" => "B",
  "\u{1D53B}" => "D",
  "\u{1D53C}" => "E",
  "\u{1D53D}" => "F",
  "\u{1D53E}" => "G",
  "\u{1D540}" => "1",
  "\u{1D541}" => "J",
  "\u{1D542}" => "K",
  "\u{1D543}" => "L",
  "\u{1D544}" => "M",
  "\u{1D546}" => "0",
  "\u{1D54A}" => "S",
  "\u{1D54B}" => "T",
  "\u{1D54C}" => "U",
  "\u{1D54D}" => "V",
  "\u{1D54E}" => "W",
  "\u{1D54F}" => "X",
  "\u{1D550}" => "Y",
  "\u{1D552}" => "a",
  "\u{1D553}" => "b",
  "\u{1D554}" => "c",
  "\u{1D555}" => "d",
  "\u{1D556}" => "e",
  "\u{1D557}" => "f",
  "\u{1D558}" => "g",
  "\u{1D559}" => "h",
  "\u{1D55A}" => "i",
  "\u{1D55B}" => "j",
  "\u{1D55C}" => "k",
  "\u{1D55D}" => "1",
  "\u{1D55E}" => "m",
  "\u{1D55F}" => "n",
  "\u{1D560}" => "o",
  "\u{1D561}" => "p",
  "\u{1D562}" => "q",
  "\u{1D563}" => "r",
  "\u{1D564}" => "s",
  "\u{1D565}" => "t",
  "\u{1D566}" => "u",
  "\u{1D567}" => "v",
  "\u{1D568}" => "w",
  "\u{1D569}" => "x",
  "\u{1D56A}" => "y",
  "\u{1D56B}" => "z",
  "\u{1D56C}" => "A",
  "\u{1D56D}" => "B",
  "\u{1D56E}" => "C",
  "\u{1D56F}" => "D",
  "\u{1D570}" => "E",
  "\u{1D571}" => "F",
  "\u{1D572}" => "G",
  "\u{1D573}" => "H",
  "\u{1D574}" => "1",
  "\u{1D575}" => "J",
  "\u{1D576}" => "K",
  "\u{1D577}" => "L",
  "\u{1D578}" => "M",
  "\u{1D579}" => "N",
  "\u{1D57A}" => "0",
  "\u{1D57B}" => "P",
  "\u{1D57C}" => "Q",
  "\u{1D57D}" => "R",
  "\u{1D57E}" => "S",
  "\u{1D57F}" => "T",
  "\u{1D580}" => "U",
  "\u{1D581}" => "V",
  "\u{1D582}" => "W",
  "\u{1D583}" => "X",
  "\u{1D584}" => "Y",
  "\u{1D585}" => "Z",
  "\u{1D586}" => "a",
  "\u{1D587}" => "b",
  "\u{1D588}" => "c",
  "\u{1D589}" => "d",
  "\u{1D58A}" => "e",
  "\u{1D58B}" => "f",
  "\u{1D58C}" => "g",
  "\u{1D58D}" => "h",
  "\u{1D58E}" => "i",
  "\u{1D58F}" => "j",
  "\u{1D590}" => "k",
  "\u{1D591}" => "1",
  "\u{1D592}" => "m",
  "\u{1D593}" => "n",
  "\u{1D594}" => "o",
  "\u{1D595}" => "p",
  "\u{1D596}" => "q",
  "\u{1D597}" => "r",
  "\u{1D598}" => "s",
  "\u{1D599}" => "t",
  "\u{1D59A}" => "u",
  "\u{1D59B}" => "v",
  "\u{1D59C}" => "w",
  "\u{1D59D}" => "x",
  "\u{1D59E}" => "y",
  "\u{1D59F}" => "z",
  "\u{1D5A0}" => "A",
  "\u{1D5A1}" => "B",
  "\u{1D5A2}" => "C",
  "\u{1D5A3}" => "D",
  "\u{1D5A4}" => "E",
  "\u{1D5A5}" => "F",
  "\u{1D5A6}" => "G",
  "\u{1D5A7}" => "H",
  "\u{1D5A8}" => "1",
  "\u{1D5A9}" => "J",
  "\u{1D5AA}" => "K",
  "\u{1D5AB}" => "L",
  "\u{1D5AC}" => "M",
  "\u{1D5AD}" => "N",
  "\u{1D5AE}" => "0",
  "\u{1D5AF}" => "P",
  "\u{1D5B0}" => "Q",
  "\u{1D5B1}" => "R",
  "\u{1D5B2}" => "S",
  "\u{1D5B3}" => "T",
  "\u{1D5B4}" => "U",
  "\u{1D5B5}" => "V",
  "\u{1D5B6}" => "W",
  "\u{1D5B7}" => "X",
  "\u{1D5B8}" => "Y",
  "\u{1D5B9}" => "Z",
  "\u{1D5BA}" => "a",
  "\u{1D5BB}" => "b",
  "\u{1D5BC}" => "c",
  "\u{1D5BD}" => "d",
  "\u{1D5BE}" => "e",
  "\u{1D5BF}" => "f",
  "\u{1D5C0}" => "g",
  "\u{1D5C1}" => "h",
  "\u{1D5C2}" => "i",
  "\u{1D5C3}" => "j",
  "\u{1D5C4}" => "k",
  "\u{1D5C5}" => "1",
  "\u{1D5C6}" => "m",
  "\u{1D5C7}" => "n",
  "\u{1D5C8}" => "o",
  "\u{1D5C9}" => "p",
  "\u{1D5CA}" => "q",
  "\u{1D5CB}" => "r",
  "\u{1D5CC}" => "s",
  "\u{1D5CD}" => "t",
  "\u{1D5CE}" => "u",
  "\u{1D5CF}" => "v",
  "\u{1D5D0}" => "w",
  "\u{1D5D1}" => "x",
  "\u{1D5D2}" => "y",
  "\u{1D5D3}" => "z",
  "\u{1D5D4}" => "A",
  "\u{1D5D5}" => "B",
  "\u{1D5D6}" => "C",
  "\u{1D5D7}" => "D",
  "\u{1D5D8}" => "E",
  "\u{1D5D9}" => "F",
  "\u{1D5DA}" => "G",
  "\u{1D5DB}" => "H",
  "\u{1D5DC}" => "1",
  "\u{1D5DD}" => "J",
  "\u{1D5DE}" => "K",
  "\u{1D5DF}" => "L",
  "\u{1D5E0}" => "M",
  "\u{1D5E1}" => "N",
  "\u{1D5E2}" => "0",
  "\u{1D5E3}" => "P",
  "\u{1D5E4}" => "Q",
  "\u{1D5E5}" => "R",
  "\u{1D5E6}" => "S",
  "\u{1D5E7}" => "T",
  "\u{1D5E8}" => "U",
  "\u{1D5E9}" => "V",
  "\u{1D5EA}" => "W",
  "\u{1D5EB}" => "X",
  "\u{1D5EC}" => "Y",
  "\u{1D5ED}" => "Z",
  "\u{1D5EE}" => "a",
  "\u{1D5EF}" => "b",
  "\u{1D5F0}" => "c",
  "\u{1D5F1}" => "d",
  "\u{1D5F2}" => "e",
  "\u{1D5F3}" => "f",
  "\u{1D5F4}" => "g",
  "\u{1D5F5}" => "h",
  "\u{1D5F6}" => "i",
  "\u{1D5F7}" => "j",
  "\u{1D5F8}" => "k",
  "\u{1D5F9}" => "1",
  "\u{1D5FA}" => "m",
  "\u{1D5FB}" => "n",
  "\u{1D5FC}" => "o",
  "\u{1D5FD}" => "p",
  "\u{1D5FE}" => "q",
  "\u{1D5FF}" => "r",
  "\u{1D600}" => "s",
  "\u{1D601}" => "t",
  "\u{1D602}" => "u",
  "\u{1D603}" => "v",
  "\u{1D604}" => "w",
  "\u{1D605}" => "x",
  "\u{1D606}" => "y",
  "\u{1D607}" => "z",
  "\u{1D608}" => "A",
  "\u{1D609}" => "B",
  "\u{1D60A}" => "C",
  "\u{1D60B}" => "D",
  "\u{1D60C}" => "E",
  "\u{1D60D}" => "F",
  "\u{1D60E}" => "G",
  "\u{1D60F}" => "H",
  "\u{1D610}" => "1",
  "\u{1D611}" => "J",
  "\u{1D612}" => "K",
  "\u{1D613}" => "L",
  "\u{1D614}" => "M",
  "\u{1D615}" => "N",
  "\u{1D616}" => "0",
  "\u{1D617}" => "P",
  "\u{1D618}" => "Q",
  "\u{1D619}" => "R",
  "\u{1D61A}" => "S",
  "\u{1D61B}" => "T",
  "\u{1D61C}" => "U",
  "\u{1D61D}" => "V",
  "\u{1D61E}" => "W",
  "\u{1D61F}" => "X",
  "\u{1D620}" => "Y",
  "\u{1D621}" => "Z",
  "\u{1D622}" => "a",
  "\u{1D623}" => "b",
  "\u{1D624}" => "c",
  "\u{1D625}" => "d",
  "\u{1D626}" => "e",
  "\u{1D627}" => "f",
  "\u{1D628}" => "g",
  "\u{1D629}" => "h",
  "\u{1D62A}" => "i",
  "\u{1D62B}" => "j",
  "\u{1D62C}" => "k",
  "\u{1D62D}" => "1",
  "\u{1D62E}" => "m",
  "\u{1D62F}" => "n",
  "\u{1D630}" => "o",
  "\u{1D631}" => "p",
  "\u{1D632}" => "q",
  "\u{1D633}" => "r",
  "\u{1D634}" => "s",
  "\u{1D635}" => "t",
  "\u{1D636}" => "u",
  "\u{1D637}" => "v",
  "\u{1D638}" => "w",
  "\u{1D639}" => "x",
  "\u{1D63A}" => "y",
  "\u{1D63B}" => "z",
  "\u{1D63C}" => "A",
  "\u{1D63D}" => "B",
  "\u{1D63E}" => "C",
  "\u{1D63F}" => "D",
  "\u{1D640}" => "E",
  "\u{1D641}" => "F",
  "\u{1D642}" => "G",
  "\u{1D643}" => "H",
  "\u{1D644}" => "1",
  "\u{1D645}" => "J",
  "\u{1D646}" => "K",
  "\u{1D647}" => "L",
  "\u{1D648}" => "M",
  "\u{1D649}" => "N",
  "\u{1D64A}" => "0",
  "\u{1D64B}" => "P",
  "\u{1D64C}" => "Q",
  "\u{1D64D}" => "R",
  "\u{1D64E}" => "S",
  "\u{1D64F}" => "T",
  "\u{1D650}" => "U",
  "\u{1D651}" => "V",
  "\u{1D652}" => "W",
  "\u{1D653}" => "X",
  "\u{1D654}" => "Y",
  "\u{1D655}" => "Z",
  "\u{1D656}" => "a",
  "\u{1D657}" => "b",
  "\u{1D658}" => "c",
  "\u{1D659}" => "d",
  "\u{1D65A}" => "e",
  "\u{1D65B}" => "f",
  "\u{1D65C}" => "g",
  "\u{1D65D}" => "h",
  "\u{1D65E}" => "i",
  "\u{1D65F}" => "j",
  "\u{1D660}" => "k",
  "\u{1D661}" => "1",
  "\u{1D662}" => "m",
  "\u{1D663}" => "n",
  "\u{1D664}" => "o",
  "\u{1D665}" => "p",
  "\u{1D666}" => "q",
  "\u{1D667}" => "r",
  "\u{1D668}" => "s",
  "\u{1D669}" => "t",
  "\u{1D66A}" => "u",
  "\u{1D66B}" => "v",
  "\u{1D66C}" => "w",
  "\u{1D66D}" => "x",
  "\u{1D66E}" => "y",
  "\u{1D66F}" => "z",
  "\u{1D670}" => "A",
  "\u{1D671}" => "B",
  "\u{1D672}" => "C",
  "\u{1D673}" => "D",
  "\u{1D674}" => "E",
  "\u{1D675}" => "F",
  "\u{1D676}" => "G",
  "\u{1D677}" => "H",
  "\u{1D678}" => "1",
  "\u{1D679}" => "J",
  "\u{1D67A}" => "K",
  "\u{1D67B}" => "L",
  "\u{1D67C}" => "M",
  "\u{1D67D}" => "N",
  "\u{1D67E}" => "0",
  "\u{1D67F}" => "P",
  "\u{1D680}" => "Q",
  "\u{1D681}" => "R",
  "\u{1D682}" => "S",
  "\u{1D683}" => "T",
  "\u{1D684}" => "U",
  "\u{1D685}" => "V",
  "\u{1D686}" => "W",
  "\u{1D687}" => "X",
  "\u{1D688}" => "Y",
  "\u{1D689}" => "Z",
  "\u{1D68A}" => "a",
  "\u{1D68B}" => "b",
  "\u{1D68C}" => "c",
  "\u{1D68D}" => "d",
  "\u{1D68E}" => "e",
  "\u{1D68F}" => "f",
  "\u{1D690}" => "g",
  "\u{1D691}" => "h",
  "\u{1D692}" => "i",
  "\u{1D693}" => "j",
  "\u{1D694}" => "k",
  "\u{1D695}" => "1",
  "\u{1D696}" => "m",
  "\u{1D697}" => "n",
  "\u{1D698}" => "o",
  "\u{1D699}" => "p",
  "\u{1D69A}" => "q",
  "\u{1D69B}" => "r",
  "\u{1D69C}" => "s",
  "\u{1D69D}" => "t",
  "\u{1D69E}" => "u",
  "\u{1D69F}" => "v",
  "\u{1D6A0}" => "w",
  "\u{1D6A1}" => "x",
  "\u{1D6A2}" => "y",
  "\u{1D6A3}" => "z",
  "\u{1D6A4}" => "i",
  "\u{1D6A8}" => "A",
  "\u{1D6A9}" => "B",
  "\u{1D6AC}" => "E",
  "\u{1D6AD}" => "Z",
  "\u{1D6AE}" => "H",
  "\u{1D6B0}" => "1",
  "\u{1D6B1}" => "K",
  "\u{1D6B3}" => "M",
  "\u{1D6B4}" => "N",
  "\u{1D6B6}" => "0",
  "\u{1D6B8}" => "P",
  "\u{1D6BB}" => "T",
  "\u{1D6BC}" => "Y",
  "\u{1D6BE}" => "X",
  "\u{1D6C2}" => "a",
  "\u{1D6C4}" => "y",
  "\u{1D6CA}" => "i",
  "\u{1D6CE}" => "v",
  "\u{1D6D0}" => "o",
  "\u{1D6D2}" => "p",
  "\u{1D6D4}" => "o",
  "\u{1D6D6}" => "u",
  "\u{1D6E0}" => "p",
  "\u{1D6E2}" => "A",
  "\u{1D6E3}" => "B",
  "\u{1D6E6}" => "E",
  "\u{1D6E7}" => "Z",
  "\u{1D6E8}" => "H",
  "\u{1D6EA}" => "1",
  "\u{1D6EB}" => "K",
  "\u{1D6ED}" => "M",
  "\u{1D6EE}" => "N",
  "\u{1D6F0}" => "0",
  "\u{1D6F2}" => "P",
  "\u{1D6F5}" => "T",
  "\u{1D6F6}" => "Y",
  "\u{1D6F8}" => "X",
  "\u{1D6FC}" => "a",
  "\u{1D6FE}" => "y",
  "\u{1D704}" => "i",
  "\u{1D708}" => "v",
  "\u{1D70A}" => "o",
  "\u{1D70C}" => "p",
  "\u{1D70E}" => "o",
  "\u{1D710}" => "u",
  "\u{1D71A}" => "p",
  "\u{1D71C}" => "A",
  "\u{1D71D}" => "B",
  "\u{1D720}" => "E",
  "\u{1D721}" => "Z",
  "\u{1D722}" => "H",
  "\u{1D724}" => "1",
  "\u{1D725}" => "K",
  "\u{1D727}" => "M",
  "\u{1D728}" => "N",
  "\u{1D72A}" => "0",
  "\u{1D72C}" => "P",
  "\u{1D72F}" => "T",
  "\u{1D730}" => "Y",
  "\u{1D732}" => "X",
  "\u{1D736}" => "a",
  "\u{1D738}" => "y",
  "\u{1D73E}" => "i",
  "\u{1D742}" => "v",
  "\u{1D744}" => "o",
  "\u{1D746}" => "p",
  "\u{1D748}" => "o",
  "\u{1D74A}" => "u",
  "\u{1D754}" => "p",
  "\u{1D756}" => "A",
  "\u{1D757}" => "B",
  "\u{1D75A}" => "E",
  "\u{1D75B}" => "Z",
  "\u{1D75C}" => "H",
  "\u{1D75E}" => "1",
  "\u{1D75F}" => "K",
  "\u{1D761}" => "M",
  "\u{1D762}" => "N",
  "\u{1D764}" => "0",
  "\u{1D766}" => "P",
  "\u{1D769}" => "T",
  "\u{1D76A}" => "Y",
  "\u{1D76C}" => "X",
  "\u{1D770}" => "a",
  "\u{1D772}" => "y",
  "\u{1D778}" => "i",
  "\u{1D77C}" => "v",
  "\u{1D77E}" => "o",
  "\u{1D780}" => "p",
  "\u{1D782}" => "o",
  "\u{1D784}" => "u",
  "\u{1D78E}" => "p",
  "\u{1D790}" => "A",
  "\u{1D791}" => "B",
  "\u{1D794}" => "E",
  "\u{1D795}" => "Z",
  "\u{1D796}" => "H",
  "\u{1D798}" => "1",
  "\u{1D799}" => "K",
  "\u{1D79B}" => "M",
  "\u{1D79C}" => "N",
  "\u{1D79E}" => "0",
  "\u{1D7A0}" => "P",
  "\u{1D7A3}" => "T",
  "\u{1D7A4}" => "Y",
  "\u{1D7A6}" => "X",
  "\u{1D7AA}" => "a",
  "\u{1D7AC}" => "y",
  "\u{1D7B2}" => "i",
  "\u{1D7B6}" => "v",
  "\u{1D7B8}" => "o",
  "\u{1D7BA}" => "p",
  "\u{1D7BC}" => "o",
  "\u{1D7BE}" => "u",
  "\u{1D7C8}" => "p",
  "\u{1D7CA}" => "F",
  "\u{1D7CE}" => "0",
  "\u{1D7CF}" => "1",
  "\u{1D7D0}" => "2",
  "\u{1D7D1}" => "3",
  "\u{1D7D2}" => "4",
  "\u{1D7D3}" => "5",
  "\u{1D7D4}" => "6",
  "\u{1D7D5}" => "7",
  "\u{1D7D6}" => "8",
  "\u{1D7D7}" => "9",
  "\u{1D7D8}" => "0",
  "\u{1D7D9}" => "1",
  "\u{1D7DA}" => "2",
  "\u{1D7DB}" => "3",
  "\u{1D7DC}" => "4",
  "\u{1D7DD}" => "5",
  "\u{1D7DE}" => "6",
  "\u{1D7DF}" => "7",
  "\u{1D7E0}" => "8",
  "\u{1D7E1}" => "9",
  "\u{1D7E2}" => "0",
  "\u{1D7E3}" => "1",
  "\u{1D7E4}" => "2",
  "\u{1D7E5}" => "3",
  "\u{1D7E6}" => "4",
  "\u{1D7E7}" => "5",
  "\u{1D7E8}" => "6",
  "\u{1D7E9}" => "7",
  "\u{1D7EA}" => "8",
  "\u{1D7EB}" => "9",
  "\u{1D7EC}" => "0",
  "\u{1D7ED}" => "1",
  "\u{1D7EE}" => "2",
  "\u{1D7EF}" => "3",
  "\u{1D7F0}" => "4",
  "\u{1D7F1}" => "5",
  "\u{1D7F2}" => "6",
  "\u{1D7F3}" => "7",
  "\u{1D7F4}" => "8",
  "\u{1D7F5}" => "9",
  "\u{1D7F6}" => "0",
  "\u{1D7F7}" => "1",
  "\u{1D7F8}" => "2",
  "\u{1D7F9}" => "3",
  "\u{1D7FA}" => "4",
  "\u{1D7FB}" => "5",
  "\u{1D7FC}" => "6",
  "\u{1D7FD}" => "7",
  "\u{1D7FE}" => "8",
  "\u{1D7FF}" => "9",
  "\u{1F100}" => "0.",
  "\u{1F101}" => "0,",
  "\u{1F102}" => "1,",
  "\u{1F103}" => "2,",
  "\u{1F104}" => "3,",
  "\u{1F105}" => "4,",
  "\u{1F106}" => "5,",
  "\u{1F107}" => "6,",
  "\u{1F108}" => "7,",
  "\u{1F109}" => "8,",
  "\u{1F10A}" => "9,",
  "\u{1F110}" => "(A)",
  "\u{1F111}" => "(B)",
  "\u{1F112}" => "(C)",
  "\u{1F113}" => "(D)",
  "\u{1F114}" => "(E)",
  "\u{1F115}" => "(F)",
  "\u{1F116}" => "(G)",
  "\u{1F117}" => "(H)",
  "\u{1F118}" => "(1)",
  "\u{1F119}" => "(J)",
  "\u{1F11A}" => "(K)",
  "\u{1F11B}" => "(L)",
  "\u{1F11C}" => "(M)",
  "\u{1F11D}" => "(N)",
  "\u{1F11E}" => "(0)",
  "\u{1F11F}" => "(P)",
  "\u{1F120}" => "(Q)",
  "\u{1F121}" => "(R)",
  "\u{1F122}" => "(S)",
  "\u{1F123}" => "(T)",
  "\u{1F124}" => "(U)",
  "\u{1F125}" => "(V)",
  "\u{1F126}" => "(W)",
  "\u{1F127}" => "(X)",
  "\u{1F128}" => "(Y)",
  "\u{1F129}" => "(Z)",
  "\u{1F12A}" => "(S)"
}.freeze
PATTERN =

Longest first, so a multi-character fold is not pre-empted by a single-character one sharing its first character.

Regexp.union(MAP.keys.sort_by { |k| -k.length }).freeze

Class Method Summary collapse

Class Method Details

.confusable?(text) ⇒ Boolean

Returns:

  • (Boolean) 


53
54
55
 
# File 'lib/vangrail/confusables.rb', line 53

def confusable?(text)
  text.to_s.match?(PATTERN)
end

.fold(text) ⇒ Object

Every word that mixes ASCII with imitators, folded. Text with no imitators at all is returned untouched and costs one match.



37
38
39
40
41
42
 
# File 'lib/vangrail/confusables.rb', line 37

def fold(text)
  body = text.to_s
  return body unless body.match?(PATTERN)

  body.gsub(WORD) { |word| mixed?(word) ? fold_word(word) : word }
end

.fold_all(text) ⇒ Object

Folds regardless of mixing. For a caller that has already decided the text should be Latin, and for measuring what the policy costs.



46
47
48
49
50
51
 
# File 'lib/vangrail/confusables.rb', line 46

def fold_all(text)
  body = text.to_s
  return body unless body.match?(PATTERN)

  fold_word(body)
end

.fold_word(word) ⇒ Object 



63
64
65
 
# File 'lib/vangrail/confusables.rb', line 63

def fold_word(word)
  word.gsub(PATTERN) { |char| MAP.fetch(char, char) }
end

.mixed?(word) ⇒ Boolean

A word imitating ASCII carries both: at least one ASCII letter or digit, and at least one character pretending to be one.

Returns:

  • (Boolean) 


59
60
61
 
# File 'lib/vangrail/confusables.rb', line 59

def mixed?(word)
  word.match?(/[A-Za-z0-9]/) && word.match?(PATTERN)
end