Bruno Haible [Fri, 31 Dec 2021 22:53:39 +0000 (23:53 +0100)]
uniwidth: Optimize.
* lib/gen-uni-tables.c (output_nonspacing_property): Add version
parameter. Emit the usual boilerplate header for generated files.
(is_width2): New function, based on lib/uniwidth/width.c.
(output_width2_property): New function.
(main): Create uniwidth/width0.h instead of uniwidth/width.c.part.
Create uniwidth/width2.h.
* lib/uniwidth/width.c: Include width0.h, width2.h, bitmap.h.
(nonspacing_table_data, nonspacing_table_ind): Remove here.
(uc_width): Replace the double-width character with an u_width2 bitmap
lookup.
* modules/uniwidth/width (Files): Add lib/uniwidth/width0.h,
lib/uniwidth/width2.h, lib/unictype/bitmap.h.
Bruno Haible [Fri, 31 Dec 2021 17:38:12 +0000 (18:38 +0100)]
Update to Unicode 14.0.0.
* lib/gen-uni-tables.c (UC_JOINING_GROUP_THIN_YEH,
UC_JOINING_GROUP_VERTICAL_TAIL): New enum values.
(fill_arabicshaping, joining_group_as_c_identifier): Recognize these
joining groups.
* lib/unictype.in.h (UC_JOINING_GROUP_THIN_YEH,
UC_JOINING_GROUP_VERTICAL_TAIL): New enum values.
* lib/unictype/joininggroup_name.h: Add the THIN_YEH, VERTICAL_TAIL
joining groups.
* lib/unictype/joininggroup_byname.gperf: Likewise.
* lib/gen-uni-tables.c (LBP_ID1, LBP_ID2): New enum values.
(LBP_ID): Assign artificial value.
(get_lbp): Use the extended_pictographic property to assign LBP_ID1,
LBP_ID2 instead of LBP_ID. Update such that unilbrk/lbrkprop.txt comes
out as expected.
(debug_output_lbp): Print either LBP_ID1 or LBP_ID2 as LBP_ID.
(lbp_value_to_string): Handle LBP_ID1, LBP_ID2 instead of LBP_ID.
(output_lbrk_rules_as_tables): Treat LBP_ID as macro that maps to two
table rows/columns. In rule LB30b, use LBP_ID2 in addition to LBP_EB.
Remove redundant part of rule LB27.
* lib/unilbrk/lbrktables.h (LBP_ID1, LBP_ID2): New enum values.
(LBP_ID): Remove enum value.
(unilbrk_table): Update declaration.
* lib/unilbrk/u8-possible-linebreaks.c (u8_possible_linebreaks_loop):
Use LBP_ID1 instead of LBP_ID.
* lib/unilbrk/u16-possible-linebreaks.c (u16_possible_linebreaks_loop):
Likewise.
* lib/unilbrk/u32-possible-linebreaks.c (u32_possible_linebreaks_loop):
Likewise.
* tests/unilbrk/test-u8-possible-linebreaks.c (test_function): Add a
test of potential future emoji.
* tests/unilbrk/test-u16-possible-linebreaks.c (test_function):
Likewise.
* tests/unilbrk/test-u32-possible-linebreaks.c (test_function):
Likewise.
* lib/uniwidth/width.c (nonspacing_table_data, nonspacing_table_ind):
Update.
(uc_width): Assign width 2 to the characters 0x1AFF0..0x1AFF3,
0x1AFF5..0x1AFFB, 0x1AFFD..0x1AFFE, 0x1B120..0x1B122, 0x1F6DD..0x1F6DF,
0x1F7F0, 0x1FA7B..0x1FA7C, 0x1FAA9..0x1FAAC, 0x1FAB7..0x1FABA,
0x1FAC3..0x1FAC5, 0x1FAD7..0x1FAD9, 0x1FAE0..0x1FAE7, 0x1FAF0..0x1FAF6.
* tests/uniwidth/test-uc_width2.sh: Expect width 0 for the characters
0x0890..0x0891, 0x0898..0x089F, 0x08CA..0x0902, 0x0C3C, 0x180F,
0x1AC1..0x1ACE, 0x1DFA, 0x10F82..0x10F85, 0x11070, 0x11073..0x11074,
0x110C2, 0x1CF00..0x1CF2D, 0x1CF30..0x1CF46, 0x1E2AE. Expect ambiguous
width for the character 0x1734. Expect width 2 for the characters
0x1AFF0..0x1AFF3, 0x1AFF5..0x1AFFB, 0x1AFFD..0x1AFFE, 0x1B120..0x1B122,
0x1F6DD..0x1F6DF, 0x1F7F0, 0x1FA7B..0x1FA7C, 0x1FAA9..0x1FAAC,
0x1FAB7..0x1FABA, 0x1FAC3..0x1FAC5, 0x1FAD7..0x1FAD9, 0x1FAE0..0x1FAE7,
0x1FAF0..0x1FAF6.
* All generated files under lib/uni* and tests/uni*: Regenerate.
* tests/uniname/NameAliases.txt: Update.
* tests/uniname/UnicodeData.txt: Update.
* tests/uninorm/NormalizationTest.txt: Update.
* tests/unigbrk/GraphemeBreakTest.txt: Update.
* tests/uniwbrk/WordBreakTest.txt: Update.
* All the affected modules: Bump required libunistring version.
Bruno Haible [Fri, 31 Dec 2021 17:15:02 +0000 (18:15 +0100)]
Correct last ChangeLog entry.
* tests/unilbrk/test-u8-possible-linebreaks.c (test_function): Add a
test for East Asian opening parenthesis.
* tests/unilbrk/test-u16-possible-linebreaks.c (test_function):
Likewise.
* tests/unilbrk/test-u32-possible-linebreaks.c (test_function):
Likewise.
Bruno Haible [Fri, 31 Dec 2021 11:20:27 +0000 (12:20 +0100)]
Update to Unicode 13.0.0.
* lib/gen-uni-tables.c (is_WBP_MIDLETTER): Add character 0x055F.
(get_wbp): Assign value WBP_ALETTER to the characters 0x02E5..0x02EB,
0x055A, 0x058A, 0xA708..0xA716.
* lib/gen-uni-tables.c (LBP_CP1, LBP_CP2, LBP_OP1, LBP_OP2): New enum
values.
(LBP_OP, LBP_CP): Assign artificial values.
(get_lbp): Use the unicode_width[] table to assign LBP_CP1, LBP_CP2
instead of LBP_CP, and LBP_OP1, LBP_OP2 instead of LBP_OP. Update such
that unilbrk/lbrkprop.txt comes out as expected.
(debug_output_lbp): Print either LBP_CP1 or LBP_CP2 as LBP_CP. Print
either LBP_OP1 or LBP_OP2 as LBP_OP.
(lbp_value_to_string): Handle LBP_CP1, LBP_CP2, LBP_OP1, LBP_OP2 instead
of LBP_CP, LBP_OP.
(output_lbrk_rules_as_tables): Treat LBP_CP and LBP_OP as macros that
map to two table rows/columns. In rule LB30, use only LBP_OP1 instead of
LBP_OP, and only LBP_CP1 instead of LBP_CP. Simplify rule LB22.
* lib/unilbrk/lbrktables.h (LBP_CP1, LBP_CP2, LBP_OP1, LBP_OP2): New
enum values.
(LBP_OP, LBP_CP): Remove enum values.
(unilbrk_table): Update declaration.
* lib/unilbrk/u8-possible-linebreaks.c (u8_possible_linebreaks_loop):
Add a test for East Asian opening parenthesis.
* lib/unilbrk/u16-possible-linebreaks.c (u16_possible_linebreaks_loop):
Likewise.
* lib/unilbrk/u32-possible-linebreaks.c (u32_possible_linebreaks_loop):
Likewise.
* lib/uniwidth/width.c (nonspacing_table_data, nonspacing_table_ind):
Update.
(uc_width): Assign width 2 to the characters 0x16FF0..0x16FF1,
0x18AF3..0x18CD5, 0x18D00..0x18D08, 0x1F6D6..0x1F6D7, 0x1F6FB..0x1F6FC,
0x1F90C, 0x1FA74, 0x1FA83..0x1FA86, 0x1FA96..0x1FAA8, 0x1FAB0..0x1FAB6,
0x1FAC0..0x1FAC2, 0x1FAD0..0x1FAD6.
Assign width 1 to the characters 0x1F93B, 0x1F946.
* tests/uniwidth/test-uc_width2.sh: Expect width 0 for the characters
0x0B55, 0x0D81, 0x1ABF..0x1AC0, 0xA82C, 0x10EAB..0x10EAC, 0x111CF,
0x1193B..0x1193C, 0x1193E, 0x11943, 0x16FE4. Expect width 2 for the
characters 0x16FF0..0x16FF1, 0x18AF3..0x18CD5, 0x18D00..0x18D08,
0x1F6D6..0x1F6D7, 0x1F6FB..0x1F6FC, 0x1F90C, 0x1FA74, 0x1FA83..0x1FA86,
0x1FA96..0x1FAA8, 0x1FAB0..0x1FAB6, 0x1FAC0..0x1FAC2, 0x1FAD0..0x1FAD6.
Expect width 1 for the characters 0x1F93B, 0x1F946.
* All generated files under lib/uni* and tests/uni*: Regenerate.
* tests/uniname/NameAliases.txt: Update.
* tests/uniname/UnicodeData.txt: Update.
* tests/uninorm/NormalizationTest.txt: Update.
* tests/unigbrk/GraphemeBreakTest.txt: Update.
* tests/uniwbrk/WordBreakTest.txt: Update.
* All the affected modules: Bump required libunistring version.
Bruno Haible [Thu, 30 Dec 2021 20:54:23 +0000 (21:54 +0100)]
Update to Unicode 12.0.0.
* lib/gen-uni-tables.c (is_property_default_ignorable_code_point):
Exclude 0x13430..0x13438.
(get_lbp): Update such that unilbrk/lbrkprop.txt comes out as expected.
(get_wbp): Map 0xFF10..0xFF19 to WBP_NUMERIC.
* lib/uniwidth/width.c (nonspacing_table_data, nonspacing_table_ind):
Update.
(uc_width): Assign width 2 to the characters 0x16FE2..0x16FE3,
0x187F2..0x187F7, 0x1B150..0x1B152, 0x1B164..0x1B167, 0x1F6D5, 0x1F6FA,
0x1F7E0..0x1F7EB, 0x1F90D..0x1F90F, 0x1FA70..0x1FA73, 0x1FA78..0x1FA7A,
0x1FA80..0x1FA82, 0x1FA90..0x1FA95.
* tests/uniwidth/test-uc_width2.sh: Expect width 0 for the characters
0x0EBA, 0xA9BD, 0x119D4..0x119D7, 0x119DA..0x119DB, 0x119E0,
0x13430..0x13438, 0x16F4F, 0x1E130..0x1E136, 0x1E2EC..0x1E2EF. Expect
width 2 for the characters 0x16FE2..0x16FE3, 0x187F2..0x187F7,
0x1B150..0x1B152, 0x1B164..0x1B167, 0x1F6D5, 0x1F6FA, 0x1F7E0..0x1F7EB,
0x1F90D..0x1F90F, 0x1FA70..0x1FA73, 0x1FA78..0x1FA7A, 0x1FA80..0x1FA82,
0x1FA90..0x1FA95.
* All generated files under lib/uni* and tests/uni*: Regenerate.
* tests/uniname/NameAliases.txt: Update.
* tests/uniname/UnicodeData.txt: Update.
* tests/uninorm/NormalizationTest.txt: Update.
* tests/unigbrk/GraphemeBreakTest.txt: Update.
* tests/uniwbrk/WordBreakTest.txt: Update.
* All the affected modules: Bump required libunistring version.
Bruno Haible [Thu, 30 Dec 2021 15:45:39 +0000 (16:45 +0100)]
Update to Unicode 11.0.0.
* lib/gen-uni-tables.c (is_property_default_ignorable_code_point):
Simplify by use of PROP_PREPENDED_CONCATENATION_MARK.
(UC_JOINING_GROUP_HANIFI_ROHINGYA_PA,
UC_JOINING_GROUP_HANIFI_ROHINGYA_KINNA_YA): New enum values.
(fill_arabicshaping, joining_group_as_c_identifier): Recognize these
joining groups.
(get_lbp): Update such that unilbrk/lbrkprop.txt comes out as expected.
(WBP_EB, WBP_EM, WBP_GAZ, WBP_EBG): Remove enum values.
(WBP_WSS): New enum value.
(get_wbp): Update such that uniwbrk/wbrkprop.txt comes out as expected.
(debug_output_wbp, fill_org_wbp, debug_output_org_wbp, output_wbp):
Update for changed enum values.
* lib/unictype.in.h (UC_JOINING_GROUP_HANIFI_ROHINGYA_*): New enum
values.
* lib/unictype/joininggroup_name.h: Add the HANIFI_ROHINGYA_* joining
groups.
* lib/unictype/joininggroup_byname.gperf: Likewise.
* lib/unigbrk.in.h: Mark 4 enum values as obsolete.
* lib/unigbrk/u-grapheme-breaks.h (FUNC): Handle emoji modifier sequence
according to Unicode 11.0.0.
* lib/unigbrk/u8-grapheme-breaks.c: Include <stdbool.h>, unictype.h.
* lib/unigbrk/u16-grapheme-breaks.c: Likewise.
* lib/unigbrk/u32-grapheme-breaks.c: Likewise.
* lib/unigbrk/uc-grapheme-breaks.c: Likewise.
* modules/unigbrk/u8-grapheme-breaks (Depends-on): Add
unictype/property-extended-pictographic, stdbool.
* modules/unigbrk/u16-grapheme-breaks (Depends-on): Likewise.
* modules/unigbrk/u32-grapheme-breaks (Depends-on): Likewise.
* modules/unigbrk/uc-grapheme-breaks (Depends-on): Likewise.
* tests/unigbrk/test-u8-grapheme-breaks.c (main): Add test for emoji
modifier / ZWJ sequence.
* tests/unigbrk/test-u16-grapheme-breaks.c (main): Likewise.
* tests/unigbrk/test-u32-grapheme-breaks.c (main): Likewise.
* tests/unigbrk/test-uc-is-grapheme-break.c: Include <stdbool.h>,
unictype.h.
(main): Update workaround logic to match the one in
lib/unigbrk/u-grapheme-breaks.h.
* modules/unigbrk/uc-is-grapheme-break-tests (Depends-on): Add
unictype/property-extended-pictographic, stdbool.
* lib/uniwbrk.in.h: Mark 4 enum values as obsolete.
(WBP_WSS): New enum value.
* lib/uniwbrk/u-wordbreaks.h (FUNC): Handle emoji ZWJ sequences and
horizontal whitespace according to Unicode 11.0.0.
* lib/uniwbrk/u8-wordbreaks.c: Include unictype.h.
* lib/uniwbrk/u16-wordbreaks.c: Likewise.
* lib/uniwbrk/u32-wordbreaks.c: Likewise.
* lib/uniwbrk/wbrktable.c (uniwbrk_prop_index, uniwbrk_table): Add a row
and column for WBP_WSS.
* lib/uniwbrk/wbrktable.h (uniwbrk_prop_index, uniwbrk_table): Update
declarations.
* modules/uniwbrk/u8-wordbreaks (Depends-on): Add
unictype/property-extended-pictographic.
* modules/uniwbrk/u16-wordbreaks (Depends-on): Likewise.
* modules/uniwbrk/u32-wordbreaks (Depends-on): Likewise.
* tests/uniwbrk/test-u8-wordbreaks.c (main): Update expected results.
* tests/uniwbrk/test-u16-wordbreaks.c (main): Likewise.
* tests/uniwbrk/test-u32-wordbreaks.c (main): Likewise.
* tests/uniwbrk/test-uc-wordbreaks.c (wordbreakproperty_to_string):
Update.
* lib/unilbrk/u8-possible-linebreaks.c (u8_possible_linebreaks_loop):
Handle ZWJ according to Unicode 11.0.0.
* lib/unilbrk/u16-possible-linebreaks.c (u16_possible_linebreaks_loop):
Likewise.
* lib/unilbrk/u32-possible-linebreaks.c (u32_possible_linebreaks_loop):
Likewise.
* lib/uniwidth/width.c (nonspacing_table_data, nonspacing_table_ind):
Update.
(uc_width): Assign width 2 to the characters 0x187ED..0x187F1, 0x1F6F9,
0x1F9E7..0x1F9FF.
* tests/uniwidth/test-uc_width2.sh: Expect width 0 for the characters
0x07FD, 0x08D3, 0x09FE, 0x0C04, 0xA8FF, 0x10D24..0x10D27,
0x10F46..0x10F50, 0x110CD, 0x111C9, 0x1133B, 0x1145E, 0x1182F..0x11837,
0x11839..0x1183A, 0x11D90..0x11D91, 0x11D95, 0x11D97, 0x11EF3..0x11EF4.
Expect width 2 for the characters 0x187ED..0x187F1, 0x1F6F9,
0x1F9E7..0x1F9FF.
* All generated files under lib/uni* and tests/uni*: Regenerate.
* tests/uniname/NameAliases.txt: Update.
* tests/uniname/UnicodeData.txt: Update.
* tests/uninorm/NormalizationTest.txt: Update.
* tests/unigbrk/GraphemeBreakTest.txt: Update.
* tests/uniwbrk/WordBreakTest.txt: Update.
* All the affected modules: Bump required libunistring version.
Bruno Haible [Thu, 30 Dec 2021 10:43:09 +0000 (11:43 +0100)]
unictype: Add Emoji properties from Unicode 11.0.0.
* lib/gen-uni-tables.c (PROP_EMOJI*, PROP_EXTENDED_PICTOGRAPHIC): New
enum values.
(fill_properties): Don't require a space between the property name and
the comment. Handle the property names from emoji-data.txt.
(is_property_emoji, is_property_emoji_presentation,
is_property_emoji_modifier, is_property_emoji_modifier_base,
is_property_emoji_component, is_property_extended_pictographic): New
declarations.
(output_properties): Emit the properties emoji, emoji_presentation,
emoji_modifier, emoji_modifier_base, emoji_component,
extended_pictographic.
(get_lbp): Use the emoji_modifier property.
(main): Expect one more argument, for the emoji-data.txt file.
* lib/unictype.in.h (UC_PROPERTY_EMOJI, UC_PROPERTY_EMOJI_PRESENTATION,
UC_PROPERTY_EMOJI_MODIFIER, UC_PROPERTY_EMOJI_MODIFIER_BASE,
UC_PROPERTY_EMOJI_COMPONENT, UC_PROPERTY_EXTENDED_PICTOGRAPHIC,
uc_is_property_emoji, uc_is_property_emoji_presentation,
uc_is_property_emoji_modifier, uc_is_property_emoji_modifier_base,
uc_is_property_emoji_component, uc_is_property_extended_pictographic):
New declarations.
* lib/unictype/pr_emoji.c: New file.
* lib/unictype/pr_emoji_presentation.c: New file.
* lib/unictype/pr_emoji_modifier.c: New file.
* lib/unictype/pr_emoji_modifier_base.c: New file.
* lib/unictype/pr_emoji_component.c: New file.
* lib/unictype/pr_extended_pictographic.c: New file.
* modules/unictype/property-emoji: New file.
* modules/unictype/property-emoji-tests: New file.
* modules/unictype/property-emoji-presentation: New file.
* modules/unictype/property-emoji-presentation-tests: New file.
* modules/unictype/property-emoji-modifier: New file.
* modules/unictype/property-emoji-modifier-tests: New file.
* modules/unictype/property-emoji-modifier-base: New file.
* modules/unictype/property-emoji-modifier-base-tests: New file.
* modules/unictype/property-emoji-component: New file.
* modules/unictype/property-emoji-component-tests: New file.
* modules/unictype/property-extended-pictographic: New file.
* modules/unictype/property-extended-pictographic-tests: New file.
* modules/unictype/property-all (Depends-on): Depend on the new modules.
Bruno Haible [Wed, 29 Dec 2021 18:54:46 +0000 (19:54 +0100)]
unilbrk: Improve support for strings that contain CR-LF sequences.
* lib/gen-uni-tables.c (LBP_CR, LBP_LF): New enum values.
(get_lbp): For '\r' and '\n', return LBP_CR and LBP_LF, respectively,
instead of LBP_BK.
(debug_output_lbp, fill_org_lbp, debug_output_org_lbp,
lbp_value_to_string): Handle LBP_CR, LBP_LF.
* lib/unilbrk/lbrkprop2.h: Regenerated.
* lib/unilbrk/lbrktables.h (LBP_CR, LBP_LF): New enum values.
* lib/unilbrk.in.h (UC_BREAK_CR_BEFORE_LF): New enum value.
(u8_possible_linebreaks, u16_possible_linebreaks,
u32_possible_linebreaks, ulc_possible_linebreaks, u8_width_linebreaks,
u16_width_linebreaks, u32_width_linebreaks, ulc_width_linebreaks):
Declare also a _v2 variant, and use a C macro to activate it by default.
* lib/unilbrk/internal.h: New file.
* lib/unilbrk/u8-possible-linebreaks.c: Include unilbrk/internal.h.
(u8_possible_linebreaks_loop): Renamed from u8_possible_linebreaks. Add
'cr' parameter.
(u8_possible_linebreaks, u8_possible_linebreaks_v2): New functions.
(main): Update.
* lib/unilbrk/u16-possible-linebreaks.c: Include unilbrk/internal.h.
(u16_possible_linebreaks_loop): Renamed from u16_possible_linebreaks.
Add 'cr' parameter.
(u16_possible_linebreaks, u16_possible_linebreaks_v2): New functions.
* lib/unilbrk/u32-possible-linebreaks.c: Include unilbrk/internal.h.
(u32_possible_linebreaks_loop): Renamed from u32_possible_linebreaks.
Add 'cr' parameter.
(u32_possible_linebreaks, u32_possible_linebreaks_v2): New functions.
* lib/unilbrk/ulc-possible-linebreaks.c: Include unilbrk/internal.h,
unilbrk/lbrktables.h.
(ulc_possible_linebreaks_internal): Renamed from
ulc_possible_linebreaks. Add 'cr' parameter.
(ulc_possible_linebreaks, ulc_possible_linebreaks_v2): New functions.
(main): Update.
* lib/unilbrk/u8-width-linebreaks.c: Include unilbrk/internal.h,
unilbrk/lbrktables.h.
(u8_width_linebreaks_internal): Renamed from u8_width_linebreaks. Add
'cr' parameter.
(u8_width_linebreaks, u8_width_linebreaks_v2): New functions.
(main): Update.
* lib/unilbrk/u16-width-linebreaks.c: Include unilbrk/internal.h,
unilbrk/lbrktables.h.
(u16_width_linebreaks_internal): Renamed from u16_width_linebreaks. Add
'cr' parameter.
(u16_width_linebreaks, u16_width_linebreaks_v2): New functions.
* lib/unilbrk/u32-width-linebreaks.c: Include unilbrk/internal.h,
unilbrk/lbrktables.h.
(u32_width_linebreaks_internal): Renamed from u32_width_linebreaks. Add
'cr' parameter.
(u32_width_linebreaks, u32_width_linebreaks_v2): New functions.
* lib/unilbrk/ulc-width-linebreaks.c: Include unilbrk/internal.h,
unilbrk/lbrktables.h.
(ulc_width_linebreaks_internal): Renamed from ulc_width_linebreaks. Add
'cr' parameter.
(ulc_width_linebreaks, ulc_width_linebreaks_v2): New functions.
(main): Update.
* modules/unilbrk/u8-possible-linebreaks (Files): Add
lib/unilbrk/internal.h.
* modules/unilbrk/u16-possible-linebreaks (Files): Likewise.
* modules/unilbrk/u32-possible-linebreaks (Files): Likewise.
* modules/unilbrk/u8-width-linebreaks (Files): Likewise.
* tests/unilbrk/test-u8-possible-linebreaks.c (test_function): New
function, extracted from main. Add a test of CR-LF handling.
(main): Invoke it twice.
* tests/unilbrk/test-u16-possible-linebreaks.c (test_function): New
function, extracted from main. Add a test of CR-LF handling.
(main): Invoke it twice.
* tests/unilbrk/test-u32-possible-linebreaks.c (test_function): New
function, extracted from main. Add a test of CR-LF handling.
(main): Invoke it twice.
* tests/unilbrk/test-ulc-possible-linebreaks.c (test_function): New
function, extracted from main.
(main): Invoke it twice.
* tests/unilbrk/test-u8-width-linebreaks.c (test_function): New
function, extracted from main.
(main): Invoke it twice.
* tests/unilbrk/test-u16-width-linebreaks.c (test_function): New
function, extracted from main.
(main): Invoke it twice.
* tests/unilbrk/test-u32-width-linebreaks.c (test_function): New
function, extracted from main.
(main): Invoke it twice.
* tests/unilbrk/test-ulc-width-linebreaks.c (test_function): New
function, extracted from main.
(main): Invoke it twice.
Bruno Haible [Wed, 29 Dec 2021 11:15:37 +0000 (12:15 +0100)]
uniwbrk: Add unit tests of CR-LF sequences.
* tests/uniwbrk/test-u8-wordbreaks.c (main): Add test of a string with
CR-LF.
* tests/uniwbrk/test-u16-wordbreaks.c (main): Likewise.
* tests/uniwbrk/test-u32-wordbreaks.c (main): Likewise.
Bruno Haible [Wed, 29 Dec 2021 11:13:28 +0000 (12:13 +0100)]
unigbrk: Add unit tests of CR-LF sequences.
* tests/unigbrk/test-u8-grapheme-breaks.c (main): Add test of a string
with CR-LF.
* tests/unigbrk/test-u16-grapheme-breaks.c (main): Likewise.
* tests/unigbrk/test-u32-grapheme-breaks.c (main): Likewise.
Paul Eggert [Tue, 28 Dec 2021 23:28:08 +0000 (15:28 -0800)]
nstrftime: improve handling of invalid formats
* lib/nstrftime.c (__strftime_internal): Without this change,
‘date +'%0_-+^#1%A'’ would output ‘%A’; with it, it outputs
something like ‘%0_-+^#1Tuesday’ which is easier to debug and to
write wrappers for (such as the nstrftime/fprintftime usage in GNU
‘date’).
Paul Eggert [Tue, 28 Dec 2021 19:38:33 +0000 (11:38 -0800)]
gettime-res: new module
* lib/gettime-res.c, modules/gettime-res: New files.
* lib/timespec.h (gettime_res): New decl.
* m4/clock_time.m4 (gl_CLOCK_TIME): Also check for clock_getres.
* m4/gettime.m4 (gl_GETTIME_RES): New macro.
Paul Eggert [Tue, 28 Dec 2021 19:38:26 +0000 (11:38 -0800)]
gettime: port better to non-POSIX C2x
C2x requires a function timespec_get; prefer that to
gettimeofday, since it has better resolution.
* lib/gettime.c (gettime): Prefer timespec_get to gettimeofday.
* m4/gettime.m4 (gl_GETTIME): Check for timespec_get.
Omit a check for gettimeofday; not needed because
the gettime module depends on the gettimeofday module.
Bruno Haible [Tue, 28 Dec 2021 11:55:48 +0000 (12:55 +0100)]
gen-uni-tables: Produce license notices suitable for Gnulib.
* lib/gen-uni-tables.c (output_library_license, output_tests_license):
Produce license notices that are consistent with the
gnulib/etc/license-notices/ files.
* All generated files under lib/uni* and tests/uni*: Regenerate.
Paul Eggert [Tue, 28 Dec 2021 09:51:56 +0000 (01:51 -0800)]
perl: let caller see whether perl was found
Problem reported for coreutils by Serge Belyshev (Bug#52844).
* m4/perl.m4 (gl_PERL): Use AC_CACHE_CHECK, so that
an invoker of gl_PERL can later inspect gl_cv_prog_perl
to see whether perl was found.
Bruno Haible [Sun, 26 Dec 2021 18:54:42 +0000 (19:54 +0100)]
Update to Unicode 10.0.0, part 3.
* lib/gen-uni-tables.c (get_lbp): Update such that unilbrk/lbrkprop.txt comes
out as expected. Use is_property_regional_indicator.
(get_wbp): Update such that uniwbrk/wbrkprop.txt comes out as expected.
Use is_property_regional_indicator.
* lib/uniname/gen-uninames.lisp (main): Ignore empty lines and comment
lines in the aliases file. Needed because since 2015-06-20, we use the
unmodified Unicode NameAliases.txt file. Remove reference to undefined
variable. Correct indentation.
Bruno Haible [Sun, 26 Dec 2021 16:48:24 +0000 (17:48 +0100)]
Update to Unicode 10.0.0.
* lib/gen-uni-tables.c (PROP_REGIONAL_INDICATOR): New enum value.
(fill_properties): Recognize property "Regional_Indicator".
(is_property_regional_indicator): New function.
(output_properties): Also output the data for regional_indicator.
(UC_JOINING_GROUP_MALAYALAM_*): New enum values.
(fill_arabicshaping, joining_group_as_c_identifier): Recognize these
joining groups.
* lib/unictype/pr_regional_indicator.c: New file.
* modules/unictype/property-regional-indicator: New file.
* modules/unictype/property-regional-indicator-tests: New file.
* modules/unictype/property-all (Depends-on): Add
unictype/property-regional-indicator.
* lib/unictype.in.h (UC_JOINING_GROUP_MALAYALAM_*): New enum values.
* lib/unictype/joininggroup_name.h: Add the MALAYALAM_* joining groups.
* lib/unictype/joininggroup_byname.gperf: Likewise.
Bruno Haible [Sat, 25 Dec 2021 13:30:57 +0000 (14:30 +0100)]
gnulib-tool: Fix handling of module libtextstyle-optional.
Reported by Paul Eggert in
<https://lists.gnu.org/archive/html/bug-gnulib/2021-12/msg00152.html>.
* gnulib-tool (func_repeat_module_in_tests): New function.
(func_emit_tests_Makefile_am, func_create_testdir): Use it for the file
list and when creating tests/Makefile.am.
Paul Eggert [Sat, 25 Dec 2021 01:21:59 +0000 (17:21 -0800)]
maint: break out '&& mv $@-t $@'
* build-aux/git-version-gen, config/argz.mk, modules/configmake:
* modules/posix_spawnp-tests, modules/snippet/link-warning:
Break out '&& mv $@-t $@' to a separate line in the Make recipe,
as this makes things a bit easier to debug.
Bruno Haible [Fri, 24 Dec 2021 18:43:11 +0000 (19:43 +0100)]
tests: Fix link errors on AIX.
* modules/fnmatch-h-c++-tests (Makefile.am): Link test-fnmatch-h-c++
with $(LIB_MBRTOWC).
* modules/glob-h-c++-tests (Makefile.am): Link test-glob-h-c++ with
$(LIB_MBRTOWC).
* modules/stdlib-c++-tests (Makefile.am): Link test-stdlib-c++ with
$(GETLOADAVG_LIBS).
* modules/sys_stat-c++-tests (Makefile.am): Link test-sys_stat-c++ with
$(LIB_MBRTOWC).
* modules/uchar-c++-tests (Makefile.am): Link test-uchar-c++ with
$(LIBUNISTRING).
Paul Eggert [Tue, 21 Dec 2021 23:24:03 +0000 (15:24 -0800)]
maint: fix __clang_major__ misspellings
The intprops.h typo fix should fix a bug in Emacs on Debian testing
with clang <https://bugs.gnu.org/52711>.
* lib/asyncsafe-spin.c, lib/intprops.h, lib/pthread-spin.c:
Fix some misspellings of ‘__clang_major__’.
I left one misspelling alone, in lib/cdefs.h,
and instead reported that bug upstream here:
https://sourceware.org/pipermail/libc-alpha/2021-December/134512.html
Bruno Haible [Tue, 21 Dec 2021 20:38:30 +0000 (21:38 +0100)]
get_ppid_of, get_progname_of: Fix runtime error on Mac OS X < 10.5.
Reported by Evan Miller <emmiller@gmail.com>
<https://lists.gnu.org/archive/html/bug-gnulib/2021-12/msg00081.html>.
* lib/get_ppid_of.c (proc_pidinfo): Declare with WEAK_IMPORT_ATTRIBUTE.
(get_ppid_of): Test whether the symbol declared weak evaluates to
non-NULL before invoking it.
* lib/get_progname_of (get_progname_of): Declare with
WEAK_IMPORT_ATTRIBUTE.
(get_ppid_of): Test whether the symbol declared weak evaluates to
non-NULL before invoking it.
Paul Eggert [Tue, 21 Dec 2021 18:43:56 +0000 (10:43 -0800)]
Move AM_V_GEN to recipe start
This is especially needed when building Emacs, as it uses
‘AM_V_GEN=@$(info GEN $@)', which means the GEN line is output by
GNU make just before the recipe runs, regardless of where AM_V_GEN
appears in the recipe. And it’s also good practice even if the
default Automake ‘AM_V_GEN=@echo GEN $@;’ is used, as it’s better
to output the GEN line consistently at the start of every recipe,
even if this precedes a preparatory command that almost always
works silently.
* modules/arpa_inet, modules/dynarray, modules/net_if:
* modules/netinet_in, modules/scratch_buffer, modules/selinux-h:
* modules/sys_file, modules/sys_ioctl, modules/sys_random:
* modules/sys_resource, modules/sys_select, modules/sys_socket:
* modules/sys_stat, modules/sys_time, modules/sys_times:
* modules/sys_types, modules/sys_uio, modules/sys_utsname:
* modules/sys_wait, modules/unicase/special-casing, top/maint.mk:
Use AM_V_GEN only at the start of each recipe.
Bruno Haible [Sun, 19 Dec 2021 11:49:16 +0000 (12:49 +0100)]
gnulib-tool: Don't insist on ACLOCAL_AMFLAGS.
Reported by Bjarni Ingi Gislason <bjarniig@rhi.hi.is> in
<https://lists.gnu.org/archive/html/bug-gnulib/2021-12/msg00112.html>.
* gnulib-tool (func_import): Mention an AC_CONFIG_MACRO_DIRS invocation
as an alternative to augmenting ACLOCAL_AMFLAGS.
(--import, --add-import, --remove-import, --update): To find the m4
directories, look also for AC_CONFIG_MACRO_DIR and AC_CONFIG_MACRO_DIRS
invocations in configure.ac.
Bruno Haible [Sat, 18 Dec 2021 19:43:20 +0000 (11:43 -0800)]
stdint: Fix handling of limits.h (regression 2021-12-16).
* modules/stdint (configure.ac): Revert last change.
* gnulib-tool (func_emit_lib_Makefile_am, func_emit_tests_Makefile_am):
If --gnu-make is used, assume that AC_SUBSTed variables for conditionals
are constructed by appending the suffix '_CONDITION'.
Paul Eggert [Thu, 16 Dec 2021 20:40:14 +0000 (12:40 -0800)]
stdint: omit duplicate gl_CONFIGURE_HEADER calls
* modules/stdint (configure.ac): Omit
‘gl_CONDITIONAL_HEADER([limits.h])’, since stdint depends on
limits-h and doing gl_CONDITIONAL_HEADER twice can result in
‘configure’ failing with “*** GL_GENERATE_LIMITS_H is not set
correctly”.
Bruno Haible [Wed, 15 Dec 2021 18:18:31 +0000 (19:18 +0100)]
Accommodate non-recursive Automake in a less hacky way.
* gnulib-tool: New option --automake-subdir.
(automake_subdir): New variable.
(func_emit_initmacro_end): Add a second argument. Use it to prefix each
object file name in *_LIBOBJS and *_LTLIBOBJS.
(func_emit_shellvars_init): New function.
(func_import): Add support for --automake-subdir. Invoke
prefix-gnulib-mk. Update calls to func_emit_initmacro_end. Call
func_emit_shellvars_init.
(func_create_testdir): Update calls to func_emit_initmacro_end. Call
func_emit_shellvars_init.
* m4/gnulib-tool.m4 (gl_AUTOMAKE_SUBDIR): New macro.
* m4/gnulib-common.m4 (gl_CONDITIONAL_HEADER): Use the value of the
gl_source_base_prefix variable.
* build-aux/prefix-gnulib-mk: New options --from-gnulib-tool, --prefix.
(contents_of_file): Renamed from contents.
(contents_of_stdin): New function.
(process): Inline and remove function.
* doc/gnulib-tool.texi (Non-recursive make): New section.
Bruno Haible [Wed, 15 Dec 2021 18:18:25 +0000 (19:18 +0100)]
Move .h file names out of the *.m4 files.
This is useful because Autoconf macros defined in *.m4 files can be
AC_REQUIREd. In the configure file, they are then expanded before
gl_INIT, i.e. at a place where it is not known whether they are required
for the lib/ or for the tests/ directory (or both).
* m4/gnulib-common.m4 (gl_CONDITIONAL_HEADER): New macro.
* m4/alloca.m4 (gl_FUNC_ALLOCA): Set GL_GENERATE_ALLOCA_H instead of
ALLOCA_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/alloca-opt (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/argz.m4 (gl_FUNC_ARGZ): Set GL_GENERATE_ARGZ_H instead of ARGZ_H.
Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/argz (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/assert_h.m4 (gl_ASSERT_H): Set GL_GENERATE_ASSERT_H instead of
ASSERT_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/assert-h (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/byteswap.m4 (gl_BYTESWAP): Set GL_GENERATE_BYTESWAP_H instead of
BYTESWAP_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/byteswap (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/errno_h.m4 (gl_HEADER_ERRNO_H): Set GL_GENERATE_ERRNO_H instead of
ERRNO_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
(gl_REPLACE_ERRNO_VALUE): Update.
* m4/perror.m4 (gl_FUNC_PERROR): Update.
* m4/strerror.m4 (gl_FUNC_STRERROR): Update.
* m4/strerror_r.m4 (gl_FUNC_STRERROR_R, gl_FUNC_STRERROR_R_WORKS):
Update.
* modules/errno (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/execinfo.m4 (gl_EXECINFO_H): Set GL_GENERATE_EXECINFO_H instead of
EXECINFO_H. Drop AC_SUBST and AM_CONDITIONAL invocations. Drop AC_LIBOBJ
invocation.
* modules/execinfo (configure.ac): Invoke gl_CONDITIONAL_HEADER and
AC_LIBOBJ.
* m4/float_h.m4 (gl_FLOAT_H): Set GL_GENERATE_FLOAT_H instead of
FLOAT_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/float (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/fnmatch_h.m4 (gl_FNMATCH_H, gl_REPLACE_FNMATCH_H): Set
GL_GENERATE_FNMATCH_H instead of FNMATCH_H. Drop AC_SUBST and
AM_CONDITIONAL invocations.
* modules/fnmatch-h (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* modules/fnmatch (configure.ac): Likewise.
* m4/getopt.m4 (gl_FUNC_GETOPT_POSIX, gl_GETOPT_SUBSTITUTE_HEADER): Set
GL_GENERATE_GETOPT_H instead of GETOPT_H. Set GL_GENERATE_GETOPT_CDEFS_H
instead of GETOPT_CDEFS_H. Drop AC_SUBST invocations.
* modules/getopt-posix (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/glob_h.m4 (gl_GLOB_H, gl_REPLACE_GLOB_H): Set GL_GENERATE_GLOB_H
instead of GLOB_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/glob-h (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* modules/glob (configure.ac): Likewise.
* m4/iconv_h.m4 (gl_ICONV_H_DEFAULTS, gl_REPLACE_ICONV_H): Set
GL_GENERATE_ICONV_H instead of ICONV_H. Drop AC_SUBST and AM_CONDITIONAL
invocations.
(gl_ICONV_H): Update.
* modules/iconv-h (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* modules/iconv_open (configure.ac): Likewise.
* m4/ieee754-h.m4 (gl_IEEE754_H): Set GL_GENERATE_IEEE754_H instead of
IEEE754_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/ieee754-h (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/libgmp.m4 (gl_LIBGMP): Set GL_GENERATE_GMP_H instead of GMP_H. Drop
AC_SUBST invocation.
* modules/libgmp (configure.ac): Invoke gl_CONDITIONAL_HEADER.
(Makefile.am): Use GL_GENERATE_GMP_H.
* m4/libtextstyle-optional.m4 (gl_LIBTEXTSTYLE_OPTIONAL): Set
GL_GENERATE_TEXTSTYLE_H instead of TEXTSTYLE_H. Drop AC_SUBST and
AM_CONDITIONAL invocations.
* modules/libtextstyle-optional (configure.ac): Invoke
gl_CONDITIONAL_HEADER.
* m4/limits-h.m4 (gl_LIMITS_H, gl_REPLACE_LIMITS_H): Set
GL_GENERATE_LIMITS_H instead of LIMITS_H. Drop AC_SUBST and
AM_CONDITIONAL invocations.
* modules/limits-h (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/monetary_h.m4 (gl_MONETARY_H): Set GL_GENERATE_MONETARY_H instead
of MONETARY_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/monetary (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/net_if_h.m4 (gl_HEADER_NET_IF): Set GL_GENERATE_NET_IF_H instead of
NET_IF_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/net_if (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/netinet_in_h.m4 (gl_HEADER_NETINET_IN): Set
GL_GENERATE_NETINET_IN_H instead of NETINET_IN_H. Drop AC_SUBST and
AM_CONDITIONAL invocations.
* modules/netinet_in (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/selinux-context-h.m4 (gl_HEADERS_SELINUX_CONTEXT_H): Set
GL_GENERATE_SELINUX_CONTEXT_H instead of SELINUX_CONTEXT_H. Drop
AC_SUBST and AM_CONDITIONAL invocations.
* m4/selinux-label-h.m4 (gl_HEADERS_SELINUX_LABEL_H): Set
GL_GENERATE_SELINUX_LABEL_H instead of SELINUX_LABEL_H. Drop AC_SUBST
and AM_CONDITIONAL invocations.
* modules/selinux-h (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/stdalign.m4 (gl_STDALIGN_H): Set GL_GENERATE_STDALIGN_H instead of
STDALIGN_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/stdalign (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/stdarg.m4 (gl_STDARG_H): Set GL_GENERATE_STDARG_H instead of
STDARG_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/stdarg (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/stdbool.m4 (gl_STDBOOL_H): Renamed from AM_STDBOOL_H. Set
GL_GENERATE_STDBOOL_H instead of STDBOOL_H. Drop AC_SUBST and
AM_CONDITIONAL invocations.
* m4/parse-datetime.m4 (gl_PARSE_DATETIME): Don't require AM_STDBOOL_H.
* modules/stdbool (configure.ac): Invoke gl_STDBOOL_H instead of
AM_STDBOOL_H. Invoke gl_CONDITIONAL_HEADER.
* m4/stddef_h.m4 (gl_STDDEF_H): Set GL_GENERATE_STDDEF_H instead of
STDDEF_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/stddef (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/stdint.m4 (gl_STDINT_H): Set GL_GENERATE_STDINT_H instead of
STDINT_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* m4/inttypes.m4 (gl_INTTYPES_PRI_SCN): Update.
* modules/stdint (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/stdnoreturn.m4 (gl_STDNORETURN_H): Set GL_GENERATE_STDNORETURN_H
instead of STDNORETURN_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/stdnoreturn (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* m4/sys_socket_h.m4 (gl_SYS_SOCKET_H): Set GL_GENERATE_SYS_SOCKET_H
instead of SYS_SOCKET_H.
* m4/sysexits.m4 (gl_SYSEXITS): Set GL_GENERATE_SYSEXITS_H instead of
SYSEXITS_H. Drop AC_SUBST and AM_CONDITIONAL invocations.
* modules/sysexits (configure.ac): Invoke gl_CONDITIONAL_HEADER.
* modules/sigsegv (configure.ac): Set GL_GENERATE_SIGSEGV_H instead of
SIGSEGV_H. Drop AC_SUBST and AM_CONDITIONAL invocations. Invoke
gl_CONDITIONAL_HEADER.
Paul Eggert [Tue, 14 Dec 2021 20:32:30 +0000 (12:32 -0800)]
renameatu: port to macOS tmpfs
Problem reported by Sudhip Nashi (Bug#52193).
* lib/renameatu.c (renameat2ish) [HAVE_RENAMEAT]: New function.
(renameatu): Use the new function, to avoid a bug when
renameatx_np fails with errno == ENOTSUP. Don’t try to support
RENAME_EXCHANGE; the old code didn’t work and nobody using using
RENAME_EXCHANGE anyway.
Paul Eggert [Mon, 13 Dec 2021 01:17:47 +0000 (17:17 -0800)]
bootstrap: refactor for GNU tar
* build-aux/bootstrap (gitignore_entries, insert_if_absent)
(insert_vc_ignore, symlink_to_dir): Define these shell functions
before including bootstrap.conf. This is for GNU tar, whose
bootstrap.conf uses symlink_to_dir.
Bruno Haible [Sun, 12 Dec 2021 20:13:54 +0000 (21:13 +0100)]
gnulib-tool: Support different basenames for the lib/,tests/ Makefiles.
* gnulib-tool: New option --tests-makefile-name.
(tests_makefile_name): New variable.
(func_import): Add support for --tests-makefile-name. Replace local
variable makefile_am with two local variables source_makefile_am,
tests_makefile_am.
Bruno Haible [Sat, 11 Dec 2021 16:43:48 +0000 (17:43 +0100)]
get_ppid_of, get_progname_of: Fix compilation error on Mac OS X < 10.5.
Reported by Ryan Schmidt <gnulib@ryandesign.com> in
<https://lists.gnu.org/archive/html/bug-gnulib/2021-12/msg00011.html>.
* lib/get_ppid_of.c: Include AvailabilityMacros.h
(get_ppid_of): Use MAC_OS_X_VERSION_MAX_ALLOWED and
MAC_OS_X_VERSION_MIN_REQUIRED.
* lib/get_progname_of: Include AvailabilityMacros.h
(get_progname_of): Use MAC_OS_X_VERSION_MAX_ALLOWED and
MAC_OS_X_VERSION_MIN_REQUIRED.
Paul Eggert [Fri, 10 Dec 2021 21:43:10 +0000 (13:43 -0800)]
backupfile: prefer signed integers
* lib/backupfile.c: Include ialloc.h instead of idx.h.
Prefer idx_t to size_t where either will do.
Use imalloc and irealloc instead of malloc and realloc.
* modules/backupfile, modules/backup-rename (Depends-on):
Depend on ialloc not idx.