]> Savannah Git Hosting - gnulib.git/commit
Update to Unicode 11.0.0.
authorBruno Haible <bruno@clisp.org>
Thu, 30 Dec 2021 15:45:39 +0000 (16:45 +0100)
committerBruno Haible <bruno@clisp.org>
Thu, 30 Dec 2021 17:20:55 +0000 (18:20 +0100)
commitecbed643ffd4e817a924a645832f73dc6a0abdd0
tree878cfb2fcf77bebdf7505194de8b2324f5a05428
parentef4c53b0329bd6ce418bebbbac3fdf8b52aeb2aa
Update to Unicode 11.0.0.

* lib/gen-uni-tables.c (is_property_default_ignorable_code_point):
Simplify by use of PROP_PREPENDED_CONCATENATION_MARK.
(UC_JOINING_GROUP_HANIFI_ROHINGYA_PA,
UC_JOINING_GROUP_HANIFI_ROHINGYA_KINNA_YA): New enum values.
(fill_arabicshaping, joining_group_as_c_identifier): Recognize these
joining groups.
(get_lbp): Update such that unilbrk/lbrkprop.txt comes out as expected.
(WBP_EB, WBP_EM, WBP_GAZ, WBP_EBG): Remove enum values.
(WBP_WSS): New enum value.
(get_wbp): Update such that uniwbrk/wbrkprop.txt comes out as expected.
(debug_output_wbp, fill_org_wbp, debug_output_org_wbp, output_wbp):
Update for changed enum values.

* lib/unictype.in.h (UC_JOINING_GROUP_HANIFI_ROHINGYA_*): New enum
values.
* lib/unictype/joininggroup_name.h: Add the HANIFI_ROHINGYA_* joining
groups.
* lib/unictype/joininggroup_byname.gperf: Likewise.

* lib/unigbrk.in.h: Mark 4 enum values as obsolete.
* lib/unigbrk/u-grapheme-breaks.h (FUNC): Handle emoji modifier sequence
according to Unicode 11.0.0.
* lib/unigbrk/u8-grapheme-breaks.c: Include <stdbool.h>, unictype.h.
* lib/unigbrk/u16-grapheme-breaks.c: Likewise.
* lib/unigbrk/u32-grapheme-breaks.c: Likewise.
* lib/unigbrk/uc-grapheme-breaks.c: Likewise.
* modules/unigbrk/u8-grapheme-breaks (Depends-on): Add
unictype/property-extended-pictographic, stdbool.
* modules/unigbrk/u16-grapheme-breaks (Depends-on): Likewise.
* modules/unigbrk/u32-grapheme-breaks (Depends-on): Likewise.
* modules/unigbrk/uc-grapheme-breaks (Depends-on): Likewise.
* tests/unigbrk/test-u8-grapheme-breaks.c (main): Add test for emoji
modifier / ZWJ sequence.
* tests/unigbrk/test-u16-grapheme-breaks.c (main): Likewise.
* tests/unigbrk/test-u32-grapheme-breaks.c (main): Likewise.
* tests/unigbrk/test-uc-is-grapheme-break.c: Include <stdbool.h>,
unictype.h.
(main): Update workaround logic to match the one in
lib/unigbrk/u-grapheme-breaks.h.
* modules/unigbrk/uc-is-grapheme-break-tests (Depends-on): Add
unictype/property-extended-pictographic, stdbool.

* lib/uniwbrk.in.h: Mark 4 enum values as obsolete.
(WBP_WSS): New enum value.
* lib/uniwbrk/u-wordbreaks.h (FUNC): Handle emoji ZWJ sequences and
horizontal whitespace according to Unicode 11.0.0.
* lib/uniwbrk/u8-wordbreaks.c: Include unictype.h.
* lib/uniwbrk/u16-wordbreaks.c: Likewise.
* lib/uniwbrk/u32-wordbreaks.c: Likewise.
* lib/uniwbrk/wbrktable.c (uniwbrk_prop_index, uniwbrk_table): Add a row
and column for WBP_WSS.
* lib/uniwbrk/wbrktable.h (uniwbrk_prop_index, uniwbrk_table): Update
declarations.
* modules/uniwbrk/u8-wordbreaks (Depends-on): Add
unictype/property-extended-pictographic.
* modules/uniwbrk/u16-wordbreaks (Depends-on): Likewise.
* modules/uniwbrk/u32-wordbreaks (Depends-on): Likewise.
* tests/uniwbrk/test-u8-wordbreaks.c (main): Update expected results.
* tests/uniwbrk/test-u16-wordbreaks.c (main): Likewise.
* tests/uniwbrk/test-u32-wordbreaks.c (main): Likewise.
* tests/uniwbrk/test-uc-wordbreaks.c (wordbreakproperty_to_string):
Update.

* lib/unilbrk/u8-possible-linebreaks.c (u8_possible_linebreaks_loop):
Handle ZWJ according to Unicode 11.0.0.
* lib/unilbrk/u16-possible-linebreaks.c (u16_possible_linebreaks_loop):
Likewise.
* lib/unilbrk/u32-possible-linebreaks.c (u32_possible_linebreaks_loop):
Likewise.

* lib/uniwidth/width.c (nonspacing_table_data, nonspacing_table_ind):
Update.
(uc_width): Assign width 2 to the characters 0x187ED..0x187F1, 0x1F6F9,
0x1F9E7..0x1F9FF.
* tests/uniwidth/test-uc_width2.sh: Expect width 0 for the characters
0x07FD, 0x08D3, 0x09FE, 0x0C04, 0xA8FF, 0x10D24..0x10D27,
0x10F46..0x10F50, 0x110CD, 0x111C9, 0x1133B, 0x1145E, 0x1182F..0x11837,
0x11839..0x1183A, 0x11D90..0x11D91, 0x11D95, 0x11D97, 0x11EF3..0x11EF4.
Expect width 2 for the characters 0x187ED..0x187F1, 0x1F6F9,
0x1F9E7..0x1F9FF.

* All generated files under lib/uni* and tests/uni*: Regenerate.
* tests/uniname/NameAliases.txt: Update.
* tests/uniname/UnicodeData.txt: Update.
* tests/uninorm/NormalizationTest.txt: Update.
* tests/unigbrk/GraphemeBreakTest.txt: Update.
* tests/uniwbrk/WordBreakTest.txt: Update.

* All the affected modules: Bump required libunistring version.
311 files changed:
ChangeLog
lib/gen-uni-tables.c
lib/unicase/cased.h
lib/unicase/ignorable.h
lib/unicase/special-casing-table.gperf
lib/unicase/tocasefold.h
lib/unicase/tolower.h
lib/unicase/totitle.h
lib/unicase/toupper.h
lib/unictype.in.h
lib/unictype/bidi_of.h
lib/unictype/blocks.h
lib/unictype/categ_C.h
lib/unictype/categ_Cc.h
lib/unictype/categ_Cf.h
lib/unictype/categ_Cn.h
lib/unictype/categ_Co.h
lib/unictype/categ_Cs.h
lib/unictype/categ_L.h
lib/unictype/categ_LC.h
lib/unictype/categ_Ll.h
lib/unictype/categ_Lm.h
lib/unictype/categ_Lo.h
lib/unictype/categ_Lt.h
lib/unictype/categ_Lu.h
lib/unictype/categ_M.h
lib/unictype/categ_Mc.h
lib/unictype/categ_Me.h
lib/unictype/categ_Mn.h
lib/unictype/categ_N.h
lib/unictype/categ_Nd.h
lib/unictype/categ_Nl.h
lib/unictype/categ_No.h
lib/unictype/categ_P.h
lib/unictype/categ_Pc.h
lib/unictype/categ_Pd.h
lib/unictype/categ_Pe.h
lib/unictype/categ_Pf.h
lib/unictype/categ_Pi.h
lib/unictype/categ_Po.h
lib/unictype/categ_Ps.h
lib/unictype/categ_S.h
lib/unictype/categ_Sc.h
lib/unictype/categ_Sk.h
lib/unictype/categ_Sm.h
lib/unictype/categ_So.h
lib/unictype/categ_Z.h
lib/unictype/categ_Zl.h
lib/unictype/categ_Zp.h
lib/unictype/categ_Zs.h
lib/unictype/categ_of.h
lib/unictype/combiningclass.h
lib/unictype/ctype_alnum.h
lib/unictype/ctype_alpha.h
lib/unictype/ctype_blank.h
lib/unictype/ctype_cntrl.h
lib/unictype/ctype_digit.h
lib/unictype/ctype_graph.h
lib/unictype/ctype_lower.h
lib/unictype/ctype_print.h
lib/unictype/ctype_punct.h
lib/unictype/ctype_space.h
lib/unictype/ctype_upper.h
lib/unictype/ctype_xdigit.h
lib/unictype/decdigit.h
lib/unictype/digit.h
lib/unictype/joininggroup_byname.gperf
lib/unictype/joininggroup_name.h
lib/unictype/joininggroup_of.h
lib/unictype/joiningtype_of.h
lib/unictype/mirror.h
lib/unictype/numeric.h
lib/unictype/pr_alphabetic.h
lib/unictype/pr_ascii_hex_digit.h
lib/unictype/pr_bidi_arabic_digit.h
lib/unictype/pr_bidi_arabic_right_to_left.h
lib/unictype/pr_bidi_block_separator.h
lib/unictype/pr_bidi_boundary_neutral.h
lib/unictype/pr_bidi_common_separator.h
lib/unictype/pr_bidi_control.h
lib/unictype/pr_bidi_embedding_or_override.h
lib/unictype/pr_bidi_eur_num_separator.h
lib/unictype/pr_bidi_eur_num_terminator.h
lib/unictype/pr_bidi_european_digit.h
lib/unictype/pr_bidi_hebrew_right_to_left.h
lib/unictype/pr_bidi_left_to_right.h
lib/unictype/pr_bidi_non_spacing_mark.h
lib/unictype/pr_bidi_other_neutral.h
lib/unictype/pr_bidi_pdf.h
lib/unictype/pr_bidi_segment_separator.h
lib/unictype/pr_bidi_whitespace.h
lib/unictype/pr_case_ignorable.h
lib/unictype/pr_cased.h
lib/unictype/pr_changes_when_casefolded.h
lib/unictype/pr_changes_when_casemapped.h
lib/unictype/pr_changes_when_lowercased.h
lib/unictype/pr_changes_when_titlecased.h
lib/unictype/pr_changes_when_uppercased.h
lib/unictype/pr_combining.h
lib/unictype/pr_composite.h
lib/unictype/pr_currency_symbol.h
lib/unictype/pr_dash.h
lib/unictype/pr_decimal_digit.h
lib/unictype/pr_default_ignorable_code_point.h
lib/unictype/pr_deprecated.h
lib/unictype/pr_diacritic.h
lib/unictype/pr_emoji.h
lib/unictype/pr_emoji_component.h
lib/unictype/pr_emoji_modifier.h
lib/unictype/pr_emoji_modifier_base.h
lib/unictype/pr_emoji_presentation.h
lib/unictype/pr_extended_pictographic.h
lib/unictype/pr_extender.h
lib/unictype/pr_format_control.h
lib/unictype/pr_grapheme_base.h
lib/unictype/pr_grapheme_extend.h
lib/unictype/pr_grapheme_link.h
lib/unictype/pr_hex_digit.h
lib/unictype/pr_hyphen.h
lib/unictype/pr_id_continue.h
lib/unictype/pr_id_start.h
lib/unictype/pr_ideographic.h
lib/unictype/pr_ids_binary_operator.h
lib/unictype/pr_ids_trinary_operator.h
lib/unictype/pr_ignorable_control.h
lib/unictype/pr_iso_control.h
lib/unictype/pr_join_control.h
lib/unictype/pr_left_of_pair.h
lib/unictype/pr_line_separator.h
lib/unictype/pr_logical_order_exception.h
lib/unictype/pr_lowercase.h
lib/unictype/pr_math.h
lib/unictype/pr_non_break.h
lib/unictype/pr_not_a_character.h
lib/unictype/pr_numeric.h
lib/unictype/pr_other_alphabetic.h
lib/unictype/pr_other_default_ignorable_code_point.h
lib/unictype/pr_other_grapheme_extend.h
lib/unictype/pr_other_id_continue.h
lib/unictype/pr_other_id_start.h
lib/unictype/pr_other_lowercase.h
lib/unictype/pr_other_math.h
lib/unictype/pr_other_uppercase.h
lib/unictype/pr_paired_punctuation.h
lib/unictype/pr_paragraph_separator.h
lib/unictype/pr_pattern_syntax.h
lib/unictype/pr_pattern_white_space.h
lib/unictype/pr_private_use.h
lib/unictype/pr_punctuation.h
lib/unictype/pr_quotation_mark.h
lib/unictype/pr_radical.h
lib/unictype/pr_sentence_terminal.h
lib/unictype/pr_soft_dotted.h
lib/unictype/pr_space.h
lib/unictype/pr_terminal_punctuation.h
lib/unictype/pr_titlecase.h
lib/unictype/pr_unassigned_code_value.h
lib/unictype/pr_unified_ideograph.h
lib/unictype/pr_uppercase.h
lib/unictype/pr_variation_selector.h
lib/unictype/pr_white_space.h
lib/unictype/pr_xid_continue.h
lib/unictype/pr_xid_start.h
lib/unictype/pr_zero_width.h
lib/unictype/scripts.h
lib/unictype/scripts_byname.gperf
lib/unictype/sy_c_ident.h
lib/unictype/sy_c_whitespace.h
lib/unictype/sy_java_ident.h
lib/unictype/sy_java_whitespace.h
lib/unigbrk.in.h
lib/unigbrk/gbrkprop.h
lib/unigbrk/u-grapheme-breaks.h
lib/unigbrk/u16-grapheme-breaks.c
lib/unigbrk/u32-grapheme-breaks.c
lib/unigbrk/u8-grapheme-breaks.c
lib/unigbrk/uc-grapheme-breaks.c
lib/unilbrk/lbrkprop1.h
lib/unilbrk/lbrkprop2.h
lib/unilbrk/lbrktables.c
lib/unilbrk/u16-possible-linebreaks.c
lib/unilbrk/u32-possible-linebreaks.c
lib/unilbrk/u8-possible-linebreaks.c
lib/uniname/uninames.h
lib/uninorm/composition-table.gperf
lib/uninorm/decomposition-table1.h
lib/uninorm/decomposition-table2.h
lib/uniwbrk.in.h
lib/uniwbrk/u-wordbreaks.h
lib/uniwbrk/u16-wordbreaks.c
lib/uniwbrk/u32-wordbreaks.c
lib/uniwbrk/u8-wordbreaks.c
lib/uniwbrk/wbrkprop.h
lib/uniwbrk/wbrktable.c
lib/uniwbrk/wbrktable.h
lib/uniwidth/width.c
modules/unictype/category-Cf
modules/unictype/category-LC
modules/unictype/category-Ll
modules/unictype/category-Lu
modules/unictype/category-No
modules/unictype/ctype-lower
modules/unictype/ctype-upper
modules/unictype/mirror
modules/unictype/property-bidi-arabic-digit
modules/unictype/property-cased
modules/unictype/property-changes-when-casefolded
modules/unictype/property-changes-when-casemapped
modules/unictype/property-changes-when-lowercased
modules/unictype/property-changes-when-titlecased
modules/unictype/property-changes-when-uppercased
modules/unictype/property-ignorable-control
modules/unictype/property-lowercase
modules/unictype/property-uppercase
modules/unictype/property-zero-width
modules/unigbrk/u16-grapheme-breaks
modules/unigbrk/u32-grapheme-breaks
modules/unigbrk/u8-grapheme-breaks
modules/unigbrk/uc-grapheme-breaks
modules/unigbrk/uc-is-grapheme-break-tests
modules/uniwbrk/u16-wordbreaks
modules/uniwbrk/u32-wordbreaks
modules/uniwbrk/u8-wordbreaks
tests/unicase/test-cased.c
tests/unicase/test-ignorable.c
tests/unicase/test-uc_tolower.c
tests/unicase/test-uc_totitle.c
tests/unicase/test-uc_toupper.c
tests/unictype/test-categ_C.c
tests/unictype/test-categ_Cf.c
tests/unictype/test-categ_Cn.c
tests/unictype/test-categ_L.c
tests/unictype/test-categ_LC.c
tests/unictype/test-categ_Ll.c
tests/unictype/test-categ_Lo.c
tests/unictype/test-categ_Lu.c
tests/unictype/test-categ_M.c
tests/unictype/test-categ_Mc.c
tests/unictype/test-categ_Mn.c
tests/unictype/test-categ_N.c
tests/unictype/test-categ_Nd.c
tests/unictype/test-categ_No.c
tests/unictype/test-categ_P.c
tests/unictype/test-categ_Po.c
tests/unictype/test-categ_S.c
tests/unictype/test-categ_Sc.c
tests/unictype/test-categ_So.c
tests/unictype/test-ctype_alnum.c
tests/unictype/test-ctype_alpha.c
tests/unictype/test-ctype_graph.c
tests/unictype/test-ctype_lower.c
tests/unictype/test-ctype_print.c
tests/unictype/test-ctype_punct.c
tests/unictype/test-ctype_upper.c
tests/unictype/test-decdigit.h
tests/unictype/test-digit.h
tests/unictype/test-joininggroup_of.h
tests/unictype/test-joiningtype_of.h
tests/unictype/test-numeric.h
tests/unictype/test-pr_alphabetic.c
tests/unictype/test-pr_bidi_arabic_digit.c
tests/unictype/test-pr_bidi_arabic_right_to_left.c
tests/unictype/test-pr_bidi_hebrew_right_to_left.c
tests/unictype/test-pr_bidi_left_to_right.c
tests/unictype/test-pr_bidi_non_spacing_mark.c
tests/unictype/test-pr_bidi_other_neutral.c
tests/unictype/test-pr_case_ignorable.c
tests/unictype/test-pr_cased.c
tests/unictype/test-pr_changes_when_casefolded.c
tests/unictype/test-pr_changes_when_casemapped.c
tests/unictype/test-pr_changes_when_lowercased.c
tests/unictype/test-pr_changes_when_titlecased.c
tests/unictype/test-pr_changes_when_uppercased.c
tests/unictype/test-pr_combining.c
tests/unictype/test-pr_currency_symbol.c
tests/unictype/test-pr_decimal_digit.c
tests/unictype/test-pr_diacritic.c
tests/unictype/test-pr_grapheme_base.c
tests/unictype/test-pr_grapheme_extend.c
tests/unictype/test-pr_grapheme_link.c
tests/unictype/test-pr_id_continue.c
tests/unictype/test-pr_id_start.c
tests/unictype/test-pr_ideographic.c
tests/unictype/test-pr_ignorable_control.c
tests/unictype/test-pr_lowercase.c
tests/unictype/test-pr_numeric.c
tests/unictype/test-pr_other_alphabetic.c
tests/unictype/test-pr_punctuation.c
tests/unictype/test-pr_sentence_terminal.c
tests/unictype/test-pr_terminal_punctuation.c
tests/unictype/test-pr_unassigned_code_value.c
tests/unictype/test-pr_unified_ideograph.c
tests/unictype/test-pr_uppercase.c
tests/unictype/test-pr_xid_continue.c
tests/unictype/test-pr_xid_start.c
tests/unictype/test-pr_zero_width.c
tests/unigbrk/GraphemeBreakTest.txt
tests/unigbrk/test-u16-grapheme-breaks.c
tests/unigbrk/test-u32-grapheme-breaks.c
tests/unigbrk/test-u8-grapheme-breaks.c
tests/unigbrk/test-uc-gbrk-prop.h
tests/unigbrk/test-uc-is-grapheme-break.c
tests/uniname/NameAliases.txt
tests/uniname/UnicodeData.txt
tests/uninorm/NormalizationTest.txt
tests/uniwbrk/WordBreakTest.txt
tests/uniwbrk/test-u16-wordbreaks.c
tests/uniwbrk/test-u32-wordbreaks.c
tests/uniwbrk/test-u8-wordbreaks.c
tests/uniwbrk/test-uc-wordbreaks.c
tests/uniwidth/test-uc_width2.sh