1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
|
<html>
<head>
<meta name="GENERATOR" content="Microsoft FrontPage 4.0">
<meta name="ProgId" content="FrontPage.Editor.Document">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<link rel="stylesheet" href="http://www.unicode.org/unicode.css" type="text/css">
<title>UnicodeData File Format</title>
</head>
<body>
<h1>UnicodeData File Format<br>
Version 3.0.1</h1>
<table border="1" cellspacing="2" cellpadding="0" height="87" width="100%">
<tr>
<td valign="TOP" width="144">Revision</td>
<td valign="TOP">3.0.1</td>
</tr>
<tr>
<td valign="TOP" width="144">Authors</td>
<td valign="TOP">Mark Davis and Ken Whistler</td>
</tr>
<tr>
<td valign="TOP" width="144">Date</td>
<td valign="TOP">2000-08-17</td>
</tr>
<tr>
<td valign="TOP" width="144">This Version</td>
<td valign="TOP"><a
href="http://www.unicode.org/Public/3.0-Update1/UnicodeData-3.0.1.html">http://www.unicode.org/Public/3.0-Update1/UnicodeData-3.0.1.html</a></td>
</tr>
<tr>
<td valign="TOP" width="144">Previous Version</td>
<td valign="TOP"><a
href="http://www.unicode.org/Public/3.0-Update/UnicodeData-3.0.0.html">http://www.unicode.org/Public/3.0-Update/UnicodeData-3.0.0.html</a></td>
</tr>
<tr>
<td valign="TOP" width="144">Latest Version</td>
<td valign="TOP"><a
href="http://www.unicode.org/Public/UNIDATA/UnicodeData.html">http://www.unicode.org/Public/UNIDATA/UnicodeData.html</a></td>
</tr>
</table>
<p align="center">Copyright © 1995-2000 Unicode, Inc. All Rights reserved.<br>
<i>For more information, including Disclamer and Limitations, see <a
href="UnicodeCharacterDatabase-3.0.1.html">UnicodeCharacterDatabase-3.0.1.html</a></i></p>
<p>This document describes the format of the UnicodeData.txt file, which is one
of the files in the Unicode Character Database. The document is divided into the
following sections:
<ul>
<li><a href="#Field Formats">Field Formats</a>
<ul>
<li><a href="#General Category">General Category</a></li>
<li><a href="#Bidirectional Category">Bidirectional Category</a></li>
<li><a href="#Character Decomposition">Character Decomposition Mapping</a></li>
<li><a href="#Canonical Combining Classes">Canonical Combining Classes</a></li>
<li><a href="#Decompositions and Normalization">Decompositions and
Normalization</a></li>
<li><a href="#Case Mappings">Case Mappings</a></li>
</ul>
</li>
<li><a href="#Property Invariants">Property Invariants</a></li>
<li><a href="#Modification History">Modification History</a></li>
</ul>
<p><b>Warning: </b>the information in this file does not completely describe the
use and interpretation of Unicode character properties and behavior. It must be
used in conjunction with the data in the other files in the <a
href="UnicodeCharacterDatabase-3.0.1.html">Unicode Character Database</a>, and
relies on the notation and definitions supplied in <i><a
href="http://www.unicode.org/unicode/standard/versions/Unicode3.0.html">The
Unicode Standard</a></i>. All chapter references are to Version 3.0 of the
standard.</p>
<h2><a name="Field Formats"></a>Field Formats</h2>
<p>The file consists of lines containing fields separated by semicolons. Each
line represents the data for one encoded character in the Unicode Standard.
Every encoded character has a data entry, with the exception of certain special
ranges, as detailed below.
<ul>
<li>There are nine special ranges of characters that are represented only by
their start and end characters, since the properties in the file are
uniform, except for code values (which are all sequential and assigned).</li>
<li>The names of CJK ideograph characters and the names and decompositions of
Hangul syllable characters are algorithmically derivable. (See the Unicode
Standard and <a href="http://www.unicode.org/unicode/reports/tr15/">Unicode
Standard Annex #15</a> for more information).</li>
<li>Surrogate code values and private use characters have no names.</li>
<li>The Private Use character outside of the BMP (U+F0000..U+FFFFD,
U+100000..U+10FFFD) are listed as distinct ranges. These correspond to surrogate pairs
where the first surrogate is in the High Surrogate Private Use section.</li>
</ul>
<p>The exact ranges represented by start and end characters are:
<ul>
<li>CJK Ideographs Extension A (U+3400 - U+4DB5)</li>
<li>CJK Ideographs (U+4E00 - U+9FA5)</li>
<li>Hangul Syllables (U+AC00 - U+D7A3)</li>
<li>Non-Private Use High Surrogates (U+D800 - U+DB7F)</li>
<li>Private Use High Surrogates (U+DB80 - U+DBFF)</li>
<li>Low Surrogates (U+DC00 - U+DFFF)</li>
<li>The Private Use Area (U+E000 - U+F8FF)</li>
<li>Plane 15 Private Use Area (U+F0000 - U+FFFFD)</li>
<li>Plane 16 Private Use Area (U+100000 - U+10FFFD)</li>
</ul>
<p>The following table describes the format and meaning of each field in a data
entry in the UnicodeData file. Fields which contain normative information are so
indicated.</p>
<table border="1" cellspacing="2" cellpadding="2">
<tr>
<th valign="top" align="LEFT">
<p align="LEFT">Field</th>
<th valign="top" align="LEFT">
<p align="LEFT">Name</th>
<th valign="top" align="LEFT">
<p align="LEFT">Status</th>
<th valign="top" align="LEFT">
<p align="LEFT">Explanation</th>
</tr>
<tr>
<th valign="top">0</th>
<td valign="top">Code value</td>
<td valign="top">normative</td>
<td valign="top">Code value. For characters in the range U+0000..U+FFFD
the code value uses a 4-digit hexadecimal format; for characters in the
range U+10000..U+FFFFD the code value uses a 5-digit hexadecimal format;
and for characters in the range U+100000..U+10FFFD the code value uses a
6-digit hexadecimal format.</td>
</tr>
<tr>
<th valign="top">1</th>
<td valign="top">Character name</td>
<td valign="top">normative</td>
<td valign="top">These names match exactly the names published in Chapter 14
of the Unicode Standard, Version 3.0.</td>
</tr>
<tr>
<th valign="top">2</th>
<td valign="top"><a href="#General Category">General Category</a></td>
<td valign="top">normative / informative<br>
(see below)</td>
<td valign="top">This is a useful breakdown into various "character
types" which can be used as a default categorization in
implementations. See below for a brief explanation.</td>
</tr>
<tr>
<th valign="top">3</th>
<td valign="top"><a href="#Canonical Combining Classes">Canonical Combining
Classes</a></td>
<td valign="top">normative</td>
<td valign="top">The classes used for the Canonical Ordering Algorithm in
the Unicode Standard. These classes are also printed in Chapter 4 of the
Unicode Standard.</td>
</tr>
<tr>
<th valign="top">4</th>
<td valign="top"><a href="#Bidirectional Category">Bidirectional Category</a></td>
<td valign="top">normative</td>
<td valign="top">See the list below for an explanation of the abbreviations
used in this field. These are the categories required by the Bidirectional
Behavior Algorithm in the Unicode Standard. These categories are
summarized in Chapter 3 of the Unicode Standard.</td>
</tr>
<tr>
<th valign="top">5</th>
<td valign="top"><a href="#Character Decomposition">Character Decomposition
Mapping</a></td>
<td valign="top">normative</td>
<td valign="top">In the Unicode Standard, not all of the mappings are full
(maximal) decompositions. Recursive application of look-up for
decompositions will, in all cases, lead to a maximal decomposition. The
decomposition mappings match exactly the decomposition mappings published
with the character names in the Unicode Standard.</td>
</tr>
<tr>
<th valign="top">6</th>
<td valign="top">Decimal digit value</td>
<td valign="top">normative</td>
<td valign="top">This is a numeric field. If the character has the decimal
digit property, as specified in Chapter 4 of the Unicode Standard, the
value of that digit is represented with an integer value in this field</td>
</tr>
<tr>
<th valign="top">7</th>
<td valign="top">Digit value</td>
<td valign="top">normative</td>
<td valign="top">This is a numeric field. If the character represents a
digit, not necessarily a decimal digit, the value is here. This covers
digits which do not form decimal radix forms, such as the compatibility
superscript digits</td>
</tr>
<tr>
<th valign="top">8</th>
<td valign="top">Numeric value</td>
<td valign="top">normative</td>
<td valign="top">This is a numeric field. If the character has the numeric
property, as specified in Chapter 4 of the Unicode Standard, the value of
that character is represented with an integer or rational number in this
field. This includes fractions as, e.g., "1/5" for U+2155 VULGAR
FRACTION ONE FIFTH Also included are numerical values for compatibility
characters such as circled numbers.</td>
</tr>
<tr>
<th valign="top">9</th>
<td valign="top">Mirrored</td>
<td valign="top">normative</td>
<td valign="top">If the character has been identified as a
"mirrored" character in bidirectional text, this field has the
value "Y"; otherwise "N". The list of mirrored
characters is also printed in Chapter 4 of the Unicode Standard.</td>
</tr>
<tr>
<th valign="top">10</th>
<td valign="top">Unicode 1.0 Name</td>
<td valign="top">informative</td>
<td valign="top">This is the old name as published in Unicode 1.0. This name
is only provided when it is significantly different from the Unicode 3.0
name for the character.</td>
</tr>
<tr>
<th valign="top">11</th>
<td valign="top">10646 comment field</td>
<td valign="top">informative</td>
<td valign="top">This is the ISO 10646 comment field. It appears in parentheses
in the 10646 names list, or contains an asterisk to mark an Annex P note.</td>
</tr>
<tr>
<th valign="top">12</th>
<td valign="top"><a href="#Case Mappings">Uppercase Mapping</a></td>
<td valign="top">informative</td>
<td valign="top">Upper case equivalent mapping. If a character is part of an
alphabet with case distinctions, and has an upper case equivalent, then
the upper case equivalent is in this field. See the explanation below on
case distinctions. These mappings are always one-to-one, not one-to-many
or many-to-one. This field is informative.</td>
</tr>
<tr>
<th valign="top">13</th>
<td valign="top"><a href="#Case Mappings">Lowercase Mapping</a></td>
<td valign="top">informative</td>
<td valign="top">Similar to Uppercase mapping</td>
</tr>
<tr>
<th valign="top">14</th>
<td valign="top"><a href="#Case Mappings">Titlecase Mapping</a></td>
<td valign="top">informative</td>
<td valign="top">Similar to Uppercase mapping</td>
</tr>
</table>
<h3><a name="General Category"></a>General Category</h3>
<p>The values in this field are abbreviations for the following. Some of the
values are normative, and some are informative. For more information, see the
Unicode Standard.</p>
<p><b>Note:</b> the standard does not assign information to control characters
(except for certain cases in the Bidirectional Algorithm). Implementations will
generally also assign categories to certain control characters, notably CR and
LF, according to platform conventions.</p>
<h4>Normative Categories</h4>
<table border="0" cellspacing="2" cellpadding="0">
<tr>
<th>
<p align="LEFT">Abbr.</th>
<th>
<p align="LEFT">Description</th>
</tr>
<tr>
<td align="CENTER">Lu</td>
<td>Letter, Uppercase</td>
</tr>
<tr>
<td align="CENTER">Ll</td>
<td>Letter, Lowercase</td>
</tr>
<tr>
<td align="CENTER">Lt</td>
<td>Letter, Titlecase</td>
</tr>
<tr>
<td align="CENTER">Mn</td>
<td>Mark, Non-Spacing</td>
</tr>
<tr>
<td align="CENTER">Mc</td>
<td>Mark, Spacing Combining</td>
</tr>
<tr>
<td align="CENTER">Me</td>
<td>Mark, Enclosing</td>
</tr>
<tr>
<td align="CENTER">Nd</td>
<td>Number, Decimal Digit</td>
</tr>
<tr>
<td align="CENTER">Nl</td>
<td>Number, Letter</td>
</tr>
<tr>
<td align="CENTER">No</td>
<td>Number, Other</td>
</tr>
<tr>
<td align="CENTER">Zs</td>
<td>Separator, Space</td>
</tr>
<tr>
<td align="CENTER">Zl</td>
<td>Separator, Line</td>
</tr>
<tr>
<td align="CENTER">Zp</td>
<td>Separator, Paragraph</td>
</tr>
<tr>
<td align="CENTER">Cc</td>
<td>Other, Control</td>
</tr>
<tr>
<td align="CENTER">Cf</td>
<td>Other, Format</td>
</tr>
<tr>
<td align="CENTER">Cs</td>
<td>Other, Surrogate</td>
</tr>
<tr>
<td align="CENTER">Co</td>
<td>Other, Private Use</td>
</tr>
<tr>
<td align="CENTER">Cn</td>
<td>Other, Not Assigned (no characters in the file have this property)</td>
</tr>
</table>
<h4>Informative Categories</h4>
<table border="0" cellspacing="2" cellpadding="0">
<tr>
<th>
<p align="LEFT">Abbr.</th>
<th>
<p align="LEFT">Description</th>
</tr>
<tr>
<td align="CENTER">Lm</td>
<td>Letter, Modifier</td>
</tr>
<tr>
<td align="CENTER">Lo</td>
<td>Letter, Other</td>
</tr>
<tr>
<td align="CENTER">Pc</td>
<td>Punctuation, Connector</td>
</tr>
<tr>
<td align="CENTER">Pd</td>
<td>Punctuation, Dash</td>
</tr>
<tr>
<td align="CENTER">Ps</td>
<td>Punctuation, Open</td>
</tr>
<tr>
<td align="CENTER">Pe</td>
<td>Punctuation, Close</td>
</tr>
<tr>
<td align="CENTER">Pi</td>
<td>Punctuation, Initial quote (may behave like Ps or Pe depending on usage)</td>
</tr>
<tr>
<td align="CENTER">Pf</td>
<td>Punctuation, Final quote (may behave like Ps or Pe depending on usage)</td>
</tr>
<tr>
<td align="CENTER">Po</td>
<td>Punctuation, Other</td>
</tr>
<tr>
<td align="CENTER">Sm</td>
<td>Symbol, Math</td>
</tr>
<tr>
<td align="CENTER">Sc</td>
<td>Symbol, Currency</td>
</tr>
<tr>
<td align="CENTER">Sk</td>
<td>Symbol, Modifier</td>
</tr>
<tr>
<td align="CENTER">So</td>
<td>Symbol, Other</td>
</tr>
</table>
<h3><a name="Bidirectional Category"></a>Bidirectional Category</h3>
<p>Please refer to Chapter 3 for an explanation of the algorithm for
Bidirectional Behavior and an explanation of the significance of these
categories. An up-to-date version can be found on <a
href="http://www.unicode.org/unicode/reports/tr9/">Unicode Standard Annex #9:
The Bidirectional Algorithm</a>. These values are normative.</p>
<table border="0" cellpadding="2">
<tr>
<th valign="TOP" align="LEFT">
<p align="LEFT">Type</th>
<th valign="TOP" align="LEFT">
<p align="LEFT">Description</th>
</tr>
<tr>
<td valign="TOP"><b>L</b></td>
<td valign="TOP">Left-to-Right</td>
</tr>
<tr>
<td valign="TOP"><b>LRE</b></td>
<td valign="TOP">Left-to-Right Embedding</td>
</tr>
<tr>
<td valign="TOP"><b>LRO</b></td>
<td valign="TOP">Left-to-Right Override</td>
</tr>
<tr>
<td valign="TOP"><b>R</b></td>
<td valign="TOP">Right-to-Left</td>
</tr>
<tr>
<td valign="TOP"><b>AL</b></td>
<td valign="TOP">Right-to-Left Arabic</td>
</tr>
<tr>
<td valign="TOP"><b>RLE</b></td>
<td valign="TOP">Right-to-Left Embedding</td>
</tr>
<tr>
<td valign="TOP"><b>RLO</b></td>
<td valign="TOP">Right-to-Left Override</td>
</tr>
<tr>
<td valign="TOP"><b>PDF</b></td>
<td valign="TOP">Pop Directional Format</td>
</tr>
<tr>
<td valign="TOP"><b>EN</b></td>
<td valign="TOP">European Number</td>
</tr>
<tr>
<td valign="TOP"><b>ES</b></td>
<td valign="TOP">European Number Separator</td>
</tr>
<tr>
<td valign="TOP"><b>ET</b></td>
<td valign="TOP">European Number Terminator</td>
</tr>
<tr>
<td valign="TOP"><b>AN</b></td>
<td valign="TOP">Arabic Number</td>
</tr>
<tr>
<td valign="TOP"><b>CS</b></td>
<td valign="TOP">Common Number Separator</td>
</tr>
<tr>
<td valign="TOP"><b>NSM</b></td>
<td valign="TOP">Non-Spacing Mark</td>
</tr>
<tr>
<td valign="TOP"><b>BN</b></td>
<td valign="TOP">Boundary Neutral</td>
</tr>
<tr>
<td valign="TOP"><b>B</b></td>
<td valign="TOP">Paragraph Separator</td>
</tr>
<tr>
<td valign="TOP"><b>S</b></td>
<td valign="TOP">Segment Separator</td>
</tr>
<tr>
<td valign="TOP"><b>WS</b></td>
<td valign="TOP">Whitespace</td>
</tr>
<tr>
<td valign="TOP"><b>ON</b></td>
<td valign="TOP">Other Neutrals</td>
</tr>
</table>
<h3><a name="Character Decomposition"></a>Character Decomposition Mapping</h3>
<p>The decomposition is a normative property of a character. The tags supplied
with certain decomposition mappings generally indicate formatting information.
Where no such tag is given, the mapping is designated as canonical. Conversely,
the presence of a formatting tag also indicates that the mapping is a
compatibility mapping and not a canonical mapping. In the absence of other
formatting information in a compatibility mapping, the tag is used to
distinguish it from canonical mappings.</p>
<p>In some instances a canonical mapping or a compatibility mapping may consist
of a single character. For a canonical mapping, this indicates that the
character is a canonical equivalent of another single character. For a
compatibility mapping, this indicates that the character is a compatibility
equivalent of another single character. The compatibility formatting tags used
are:</p>
<table border="0" cellspacing="2" cellpadding="0">
<tr>
<th>Tag</th>
<th>
<p align="LEFT">Description</th>
</tr>
<tr>
<td align="CENTER"><font> </td>
<td>A font variant (e.g. a blackletter form).</td>
</tr>
<tr>
<td align="CENTER"><noBreak> </td>
<td>A no-break version of a space or hyphen.</td>
</tr>
<tr>
<td align="CENTER"><initial> </td>
<td>An initial presentation form (Arabic).</td>
</tr>
<tr>
<td align="CENTER"><medial> </td>
<td>A medial presentation form (Arabic).</td>
</tr>
<tr>
<td align="CENTER"><final> </td>
<td>A final presentation form (Arabic).</td>
</tr>
<tr>
<td align="CENTER"><isolated> </td>
<td>An isolated presentation form (Arabic).</td>
</tr>
<tr>
<td align="CENTER"><circle> </td>
<td>An encircled form.</td>
</tr>
<tr>
<td align="CENTER"><super> </td>
<td>A superscript form.</td>
</tr>
<tr>
<td align="CENTER"><sub> </td>
<td>A subscript form.</td>
</tr>
<tr>
<td align="CENTER"><vertical> </td>
<td>A vertical layout presentation form.</td>
</tr>
<tr>
<td align="CENTER"><wide> </td>
<td>A wide (or zenkaku) compatibility character.</td>
</tr>
<tr>
<td align="CENTER"><narrow> </td>
<td>A narrow (or hankaku) compatibility character.</td>
</tr>
<tr>
<td align="CENTER"><small> </td>
<td>A small variant form (CNS compatibility).</td>
</tr>
<tr>
<td align="CENTER"><square> </td>
<td>A CJK squared font variant.</td>
</tr>
<tr>
<td align="CENTER"><fraction> </td>
<td>A vulgar fraction form.</td>
</tr>
<tr>
<td align="CENTER"><compat> </td>
<td>Otherwise unspecified compatibility character.</td>
</tr>
</table>
<p><b>Reminder: </b>There is a difference between decomposition and
decomposition mapping. The decomposition mappings are defined in the UnicodeData,
while the decomposition (also termed "full decomposition") is defined
in Chapter 3 to use those mappings <i>recursively.</i>
<ul>
<li>The canonical decomposition is formed by recursively applying the
canonical mappings, then applying the canonical reordering algorithm.</li>
<li>The compatibility decomposition is formed by recursively applying the
canonical <em>and</em> compatibility mappings, then applying the canonical
reordering algorithm.</li>
</ul>
<h3><a name="Canonical Combining Classes"></a>Canonical Combining Classes</h3>
<table border="0" cellspacing="2" cellpadding="0">
<tr>
<th>
<p align="LEFT">Value</th>
<th>
<p align="LEFT">Description</th>
</tr>
<tr>
<td align="RIGHT">0:</td>
<td>Spacing, split, enclosing, reordrant, and Tibetan subjoined</td>
</tr>
<tr>
<td align="RIGHT">1:</td>
<td>Overlays and interior</td>
</tr>
<tr>
<td align="RIGHT">7:</td>
<td>Nuktas</td>
</tr>
<tr>
<td align="RIGHT">8:</td>
<td>Hiragana/Katakana voicing marks</td>
</tr>
<tr>
<td align="RIGHT">9:</td>
<td>Viramas</td>
</tr>
<tr>
<td align="RIGHT">10:</td>
<td>Start of fixed position classes</td>
</tr>
<tr>
<td align="RIGHT">199:</td>
<td>End of fixed position classes</td>
</tr>
<tr>
<td align="RIGHT">200:</td>
<td>Below left attached</td>
</tr>
<tr>
<td align="RIGHT">202:</td>
<td>Below attached</td>
</tr>
<tr>
<td align="RIGHT">204:</td>
<td>Below right attached</td>
</tr>
<tr>
<td align="RIGHT">208:</td>
<td>Left attached (reordrant around single base character)</td>
</tr>
<tr>
<td align="RIGHT">210:</td>
<td>Right attached</td>
</tr>
<tr>
<td align="RIGHT">212:</td>
<td>Above left attached</td>
</tr>
<tr>
<td align="RIGHT">214:</td>
<td>Above attached</td>
</tr>
<tr>
<td align="RIGHT">216:</td>
<td>Above right attached</td>
</tr>
<tr>
<td align="RIGHT">218:</td>
<td>Below left</td>
</tr>
<tr>
<td align="RIGHT">220:</td>
<td>Below</td>
</tr>
<tr>
<td align="RIGHT">222:</td>
<td>Below right</td>
</tr>
<tr>
<td align="RIGHT">224:</td>
<td>Left (reordrant around single base character)</td>
</tr>
<tr>
<td align="RIGHT">226:</td>
<td>Right</td>
</tr>
<tr>
<td align="RIGHT">228:</td>
<td>Above left</td>
</tr>
<tr>
<td align="RIGHT">230:</td>
<td>Above</td>
</tr>
<tr>
<td align="RIGHT">232:</td>
<td>Above right</td>
</tr>
<tr>
<td align="RIGHT">233:</td>
<td>Double below</td>
</tr>
<tr>
<td align="RIGHT">234:</td>
<td>Double above</td>
</tr>
<tr>
<td align="RIGHT">240:</td>
<td>Below (iota subscript)</td>
</tr>
</table>
<p><strong>Note: </strong>some of the combining classes in this list do not
currently have members but are specified here for completeness.</p>
<h3><a name="Decompositions and Normalization"></a>Decompositions and
Normalization</h3>
<p>Decomposition is specified in Chapter 3. <a
href="http://www.unicode.org/unicode/reports/tr15/"><i>Unicode Standard Annex
#15: Unicode Normalization Forms</i></a> specifies the interaction between decomposition
and normalization. The most up-to-date version is found on <a
href="http://www.unicode.org/unicode/reports/tr15/">http://www.unicode.org/unicode/reports/tr15/</a>.
That report specifies how the decompositions defined in UnicodeData.txt are used
to derive normalized forms of Unicode text.</p>
<p>Note that as of the 2.1.9 update of the Unicode Character Database, the
decompositions in the UnicodeData.txt file can be used to recursively derive the
full decomposition in canonical order, without the need to separately apply
canonical reordering. However, canonical reordering of combining character
sequences must still be applied in decomposition when normalizing source text
which contains any combining marks.</p>
<h3><a name="Case Mappings"></a>Case Mappings</h3>
<p>The case mapping is an informative, default mapping. Case itself, on the
other hand, has normative status. Thus, for example, 0041 LATIN CAPITAL LETTER A
is normatively uppercase, but its lowercase mapping the 0061 LATIN SMALL LETTER
A is informative. The reason for this is that case can be considered to be an
inherent property of a particular character (and is usually, but not always,
derivable from the presence of the terms "CAPITAL" or
"SMALL" in the character name), but case mappings between characters
are occasionally influenced by local conventions. For example, certain
languages, such as Turkish, German, French, or Greek may have small deviations
from the default mappings listed in UnicodeData.</p>
<p>In addition to uppercase and lowercase, because of the inclusion of certain
composite characters for compatibility, such as 01F1 LATIN CAPITAL LETTER DZ,
there is a third case, called <i>titlecase</i>, which is used where the first
letter of a word is to be capitalized (e.g. UPPERCASE, Titlecase, lowercase). An
example of such a titlecase letter is 01F2 LATIN CAPITAL LETTER D WITH SMALL
LETTER Z.</p>
<p>The uppercase, titlecase and lowercase fields are only included for
characters that have a single corresponding character of that type. Composite
characters (such as "339D SQUARE CM") that do not have a single
corresponding character of that type can be cased by decomposition.</p>
<p>For compatibility with existing parsers, UnicodeData only contains case
mappings for characters where they are one-to-one mappings; it also omits
information about context-sensitive case mappings. Information about these
special cases can be found in a separate data file, <a
href="http://www.unicode.org/Public/3.0-Update1/">SpecialCasing.txt</a>, which
has been added starting with the 2.1.8 update to the Unicode data files.
SpecialCasing.txt contains additional informative case mappings that are either
not one-to-one or which are context-sensitive.</p>
<h2><a name="Property Invariants"></a>Property Invariants</h2>
<p>Values in UnicodeData.txt are subject to correction as errors are found;
however, some characteristics of the categories themselves can be considered
invariants. Applications may wish to take these invariants into account when
choosing how to implement character properties. The following is a partial list
of known invariants for the Unicode Character Database.</p>
<h4>Database Fields</h4>
<ul>
<li>The number of fields in UnicodeData.txt is fixed.</li>
<li>The order of the fields is also fixed.
<ul>
<li>Any additional information about character properties to be added in
the future will appear in separate data tables, rather than being added
on to the existing table or by subdivision or reinterpretation of
existing fields.</li>
</ul>
</li>
</ul>
<h4>General Category</h4>
<ul>
<li>There will never be more than 32 General Category values.
<ul>
<li>It is very unlikely that the Unicode Technical Committee will
subdivide the General Category partition any further, since that can
cause implementations to misbehave. Because the General Category is
limited to 32 values, 5 bits can be used to represent the information,
and a 32-bit integer can be used as a bitmask to represent arbitrary
sets of categories.</li>
</ul>
</li>
</ul>
<h4>Combining Classes</h4>
<ul>
<li>Combining classes are limited to the values 0 to 255.
<ul>
<li>In practice, there are far fewer than 256 values used. Implementations
may take advantage of this fact for compression, since only the ordering
of the non-zero values matters for the Canonical Reordering Algorithm.
It is possible for up to 256 values to be used in the future; however,
UTC decisions in the future may restrict the number of values to 128,
since this has implementation advantages. [Signed bytes can be used
without widening to ints in Java, for example.]</li>
</ul>
</li>
<li>All characters other than those of General Category M* have the combining
class 0.
<ul>
<li>Currently, all characters other than those of General Category Mn have
the value 0. However, some characters of General Category Me or Mc may
be given non-zero values in the future.</li>
<li>The precise values above the value 0 are not invariant--only the
relative ordering is considered normative. For example, it is not
guaranteed in future versions that the class of U+05B4 will be precisely
14.</li>
</ul>
</li>
</ul>
<h4>Case</h4>
<ul>
<li>Characters of type Lu, Lt, or Ll are called <i>cased</i>. All characters
with an Upper, Lower, or Titlecase mapping are cased characters.
<ul>
<li>However, characters with the General Categories of Lu, Ll, or Lt may
not always have case mappings, and case mappings may vary by locale.
(See http://www.unicode.org/Public/UNIDATA/SpecialCasing.txt).</li>
</ul>
</li>
</ul>
<h4>Canonical Decomposition</h4>
<ul>
<li>Canonical mappings are always in canonical order.</li>
<li>Canonical mappings have only the first of a pair possibly further
decomposing.</li>
<li>Canonical decompositions are "transparent" to other character
data:
<ul>
<li><tt>BIDI(a) = BIDI(principal(canonicalDecomposition(a))</tt></li>
<li><tt>Category(a) = Category(principal(canonicalDecomposition(a))</tt></li>
<li><tt>CombiningClass(a) =
CombiningClass(principal(canonicalDecomposition(a))</tt><br>
where principal(a) is the first character not of type Mn, or the first
character if all characters are of type Mn.</li>
</ul>
</li>
<li>However, because there are sometimes missing case pairs, and because of
some legacy characters, it is only generally true that:
<ul>
<li><tt>upper(canonicalDecomposition(a)) = canonicalDecomposition(upper(a))</tt></li>
<li><tt>lower(canonicalDecomposition(a)) = canonicalDecomposition(lower(a))</tt></li>
<li><tt>title(canonicalDecomposition(a)) = canonicalDecomposition(title(a))</tt></li>
</ul>
</li>
</ul>
<h2><a name="Modification History"></a>Modification History</h2>
<p>This section provides a summary of the changes between update versions of the
Unicode Standard.</p>
<h3><a
href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 3.0.1">Unicode
3.0.1</a></h3>
<p>Modifications made for Version 3.0.1 of UnicodeData.txt include:
<ul>
<li>Added 5- and 6-digit representation of code points past U+FFFF.</li>
<li>Added Private Use range definitions for Planes 15 and 16.</li>
<li>Minor additions for the 10646 comment field.</li>
</ul>
<h3><a
href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 3.0.0">Unicode
3.0.0</a></h3>
<p>Modifications made for Version 3.0.0 of UnicodeData.txt include many new
characters and a number of property changes. These are summarized in Appendex D
of <em>The Unicode Standard, Version 3.0.</em></p>
<h3><a
href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 2.1.9">Unicode
2.1.9</a></h3>
<p>Modifications made for Version 2.1.9 of UnicodeData.txt include:
<ul>
<li>Corrected combining class for U+05AE HEBREW ACCENT ZINOR.</li>
<li>Corrected combining class for U+20E1 COMBINING LEFT RIGHT ARROW ABOVE</li>
<li>Corrected combining class for U+0F35 and U+0F37 to 220.</li>
<li>Corrected combining class for U+0F71 to 129.</li>
<li>Added a decomposition for U+0F0C TIBETAN MARK DELIMITER TSHEG BSTAR.</li>
<li>Added decompositions for several Greek symbol letters:
U+03D0..U+03D2, U+03D5, U+03D6, U+03F0..U+03F2.</li>
<li>Removed decompositions from the conjoining jamo block:
U+1100..U+11F8.</li>
<li>Changes to decomposition mappings for some Tibetan vowels for consistency
in normalization. (U+0F71, U+0F73, U+0F77, U+0F79, U+0F81)</li>
<li>Updated the decomposition mappings for several Vietnamese characters with
two diacritics (U+1EAC, U+1EAD, U+1EB6, U+1EB7, U+1EC6, U+1EC7, U+1ED8,
U+1ED9), so that the recursive decomposition can be generated directly in
canonically reordered form (not a normative change).</li>
<li>Updated the decomposition mappings for several Arabic compatibility
characters involving shadda (U+FC5E..U+FC62, U+FCF2..U+FCF4), and two Latin
characters (U+1E1C, U+1E1D), so that the decompositions are generated
directly in canonically reordered form (not a normative change).</li>
<li>Changed BIDI category for: U+00A0 NO-BREAK SPACE, U+2007 FIGURE SPACE,
U+2028 LINE SEPARATOR.</li>
<li>Changed BIDI category for extenders of General Category Lm: U+3005,
U+3021..U+3035, U+FF9E, U+FF9F.</li>
<li>Changed General Category and BIDI category for the Greek numeral signs:
U+0374, U+0375.</li>
<li>Corrected General Category for U+FFE8 HALFWIDTH FORMS LIGHT VERTICAL.</li>
<li>Added Unicode 1.0 names for many Tibetan characters (informative).</li>
</ul>
<h3><a
href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 2.1.8">Unicode
2.1.8</a></h3>
<p>Modifications made for Version 2.1.8 of UnicodeData.txt include:
<ul>
<li>Added combining class 240 for U+0345 COMBINING GREEK YPOGEGRAMMENI so that
decompositions involving iota subscript are derivable directly in
canonically reordered form; this also has a bearing on simplification of
casing of polytonic Greek.</li>
<li>Changes in decompositions related to Greek tonos. These result from the
clarification that monotonic Greek "tonos" should be equated with
U+0301 COMBINING ACUTE, rather than with U+030D COMBINING VERTICAL LINE
ABOVE. (All Greek characters in the Greek block involving "tonos";
some Greek characters in the polytonic Greek in the 1FXX block.)</li>
<li>Changed decompositions involving dialytika tonos. (U+0390, U+03B0)</li>
<li>Changed ternary decompositions to binary. (U+0CCB, U+FB2C, U+FB2D) These
changes simplify normalization.</li>
<li>Removed canonical decomposition for Latin Candrabindu. (U+0310)</li>
<li>Corrected error in canonical decomposition for U+1FF4.</li>
<li>Added compatibility decompositions to clarify collation tables. (U+2100,
U+2101, U+2105, U+2106, U+1E9A)</li>
<li>A series of general category changes to assist the convergence of of
Unicode definition of identifier with ISO TR 10176:
<ul>
<li>So > Lo: U+0950, U+0AD0, U+0F00, U+0F88..U+0F8B</li>
<li>Po > Lo: U+0E2F, U+0EAF, U+3006</li>
<li>Lm > Sk: U+309B, U+309C</li>
<li>Po > Pc: U+30FB, U+FF65</li>
<li>Ps/Pe > Mn: U+0F3E, U+0F3F</li>
</ul>
</li>
<li>A series of bidi property changes for consistency.
<ul>
<li>L > ET: U+09F2, U+09F3</li>
<li>ON > L: U+3007</li>
<li>L > ON: U+0F3A..U+0F3D, U+037E, U+0387</li>
</ul>
</li>
<li>Add case mapping: U+01A6 <-> U+0280</li>
<li>Updated symmetric swapping value for guillemets: U+00AB, U+00BB, U+2039,
U+203A.</li>
<li>Changes to combining class values. Most Indic fixed position class
non-spacing marks were changed to combining class 0. This fixes some
inconsistencies in how canonical reordering would apply to Indic scripts,
including Tibetan. Indic interacting top/bottom fixed position classes were
merged into single (non-zero) classes as part of this change. Tibetan
subjoined consonants are changed from combining class 6 to combining class
0. Thai pinthu (U+0E3A) moved to combining class 9. Moved two Devanagari
stress marks into generic above and below combining classes (U+0951,
U+0952).</li>
<li>Corrected placement of semicolon near symmetric swapping field. (U+FA0E,
etc., scattered positions to U+FA29)</li>
</ul>
<h3>Version 2.1.7</h3>
<p><i>This version was for internal change tracking only, and never publicly
released.</i></p>
<h3>Version 2.1.6</h3>
<p><i>This version was for internal change tracking only, and never publicly
released.</i></p>
<h3><a
href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 2.1.5">Unicode
2.1.5</a></h3>
<p>Modifications made for Version 2.1.5 of UnicodeData.txt include:
<ul>
<li>Changed decomposition for U+FF9E and U+FF9F so that correct collation
weighting will automatically result from the canonical equivalences.</li>
<li>Removed canonical decompositions for U+04D4, U+04D5, U+04D8, U+04D9,
U+04E0, U+04E1, U+04E8, U+04E9 (the implication being that no canonical
equivalence is claimed between these 8 characters and similar Latin
letters), and updated 4 canonical decompositions for U+04DB, U+04DC, U+04EA,
U+04EB to reflect the implied difference in the base character.</li>
<li>Added Pi, and Pf categories and assigned the relevant quotation marks to
those categories, based on the Unicode Technical Corrigendum on Quotation
Characters.</li>
<li>Updating of many bidi properties, following the advice of the ad hoc
committee on bidi, and to make the bidi properties of compatibility
characters more consistent.</li>
<li>Changed category of several Tibetan characters: U+0F3E, U+0F3F,
U+0F88..U+0F8B to make them non-combining, reflecting the combined opinion
of Tibetan experts.</li>
<li>Added case mapping for U+03F2.</li>
<li>Corrected case mapping for U+0275.</li>
<li>Added titlecase mappings for U+03D0, U+03D1, U+03D5, U+03D6, U+03F0..
U+03F2.</li>
<li>Corrected compatibility label for U+2121.</li>
<li>Add specific entries for all the CJK compatibility ideographs,
U+F900..U+FA2D, so the canonical decomposition for each (the URO character
it is equivalent to) can be carried in the database.</li>
</ul>
<h3>Version 2.1.4</h3>
<p><i>This version was for internal change tracking only, and never publicly
released.</i></p>
<h3>Version 2.1.3</h3>
<p><i>This version was for internal change tracking only, and never publicly
released.</i></p>
<h3><a
href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 2.1.2">Unicode
2.1.2</a></h3>
<p>Modifications made in updating UnicodeData.txt to Version 2.1.2 for the
Unicode Standard, Version 2.1 (from Version 2.0) include:
<ul>
<li>Added two characters (U+20AC and U+FFFC).</li>
<li>Amended bidi properties for U+0026, U+002E, U+0040, U+2007.</li>
<li>Corrected case mappings for U+018E, U+019F, U+01DD, U+0258, U+0275,
U+03C2, U+1E9B.</li>
<li>Changed combining order class for U+0F71.</li>
<li>Corrected canonical decompositions for U+0F73, U+1FBE.</li>
<li>Changed decomposition for U+FB1F from compatibility to canonical.</li>
<li>Added compatibility decompositions for U+FBE8, U+FBE9, U+FBF9..U+FBFB.</li>
<li>Corrected compatibility decompositions for U+2469, U+246A, U+3358.</li>
</ul>
<h3>Version 2.1.1</h3>
<p><i>This version was for internal change tracking only, and never publicly
released.</i></p>
<h3><a
href="http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode 2.0.0">Unicode
2.0.0</a></h3>
<p>The modifications made in updating UnicodeData.txt for the Unicode Standard,
Version 2.0 include:
<ul>
<li>Fixed decompositions with TONOS to use correct NSM: 030D.</li>
<li>Removed old Hangul Syllables; mapping to new characters are in a separate
table.</li>
<li>Marked compatibility decompositions with additional tags.</li>
<li>Changed old tag names for clarity.</li>
<li>Revision of decompositions to use first-level decomposition, instead of
maximal decomposition.</li>
<li>Correction of all known errors in decompositions from earlier versions.</li>
<li>Added control code names (as old Unicode names).</li>
<li>Added Hangul Jamo decompositions.</li>
<li>Added Number category to match properties list in book.</li>
<li>Fixed categories of Koranic Arabic marks.</li>
<li>Fixed categories of precomposed characters to match decomposition where
possible.</li>
<li>Added Hebrew cantillation marks and the Tibetan script.</li>
<li>Added place holders for ranges such as CJK Ideographic Area and the
Private Use Area.</li>
<li>Added categories Me, Sk, Pc, Nl, Cs, Cf, and rectified a number of
mistakes in the database.</li>
</ul>
</body>
</html>
|
