Paul Eggert [Wed, 19 Jul 2023 20:51:55 +0000 (13:51 -0700)]
Don’t worry about Version 7 tolower
Some code ported back to pre-C89 libraries where tolower (C) had
undefined behavior if C is not an upper case character.
Nowadays that function is _tolower which is itself obsolete,
and much Gnulib code already assumes this part of C89 anyway.
Assume C89 or better tolower, which simplifies the code
and should improve performance slightly.
* lib/mbmemcasecmp.c, lib/mbmemcasecoll.c, lib/mbscasecmp.c:
* lib/mbscasestr.c, lib/mbsncasecmp.c, lib/mbspcasecmp.c:
* lib/strcasecmp.c, lib/strcasestr.c, lib/strncasecmp.c:
(TOLOWER): Remove. All uses replaced by tolower.
Paul Eggert [Wed, 19 Jul 2023 03:33:41 +0000 (20:33 -0700)]
mbiterf, mbuiterf: port to strict C17
I ran into this problem on NetBSD 9.3.
* lib/mbiterf.h (mbiterf_next) [!GNULIB_MBRTOC32_REGULAR]:
* lib/mbuiterf.h (mbuiterf_next) [!GNULIB_MBRTOC32_REGULAR]:
Don’t label a declaration.
Bruno Haible [Tue, 18 Jul 2023 11:24:35 +0000 (13:24 +0200)]
mbuiterf: New module.
* lib/mbuiterf.h: New file, based on lib/mbuiter.h.
* lib/mbuiterf.c: New file, based on lib/mbuiter.c.
* modules/mbuiterf: New file, based on modules/mbuiter.
Bruno Haible [Tue, 18 Jul 2023 10:13:57 +0000 (12:13 +0200)]
mbiterf: New module.
* lib/mbiterf.h: New file, based on lib/mbiter.h.
* lib/mbiterf.c: New file, based on lib/mbiter.c.
* modules/mbiterf: New file, based on modules/mbiter.
Bruno Haible [Mon, 17 Jul 2023 18:50:30 +0000 (20:50 +0200)]
mbuiter: Optimize.
* lib/mbuiter.h (struct mbuiter_multi): Add cur_max field.
(mbui_init): Initialize it.
(mbuiter_multi_next): Use it instead of MB_CUR_MAX.
(mbuiter_multi_copy): Update.
Bruno Haible [Mon, 17 Jul 2023 17:11:38 +0000 (19:11 +0200)]
mbchar: Reduce size of 'struct mbchar'.
* modules/mbfile (configure.ac): Define GNULIB_MBFILE as an indicator.
* lib/mbchar.h (MBCHAR_BUF_SIZE): Set to 4.
(struct mbchar): Disable member 'buf' if the module 'mbfile' is not in
use.
Bruno Haible [Mon, 17 Jul 2023 16:00:24 +0000 (18:00 +0200)]
mbszero: Source code tweaks.
Suggested by Paul Eggert in
<https://lists.gnu.org/archive/html/bug-gnulib/2023-07/msg00084.html>.
* lib/wchar.in.h: Comment changes.
(_GL_MBSTATE_INIT_SIZE): Don't define on NetBSD, Solaris, native
Windows. Use a safe default at the end.
(_GL_MBSTATE_ZERO_SIZE): Don't define on AIX, IRIX, Solaris, native
Windows. Use a safe default at the end.
* lib/mbrtoc16.c: Update comments accordingly.
Bruno Haible [Thu, 13 Jul 2023 20:59:48 +0000 (22:59 +0200)]
localcharset: Clean up locale encodings used by glibc.
* lib/localcharset.h: Remove VISCII from the list, since never supported
in glibc/localedata/SUPPORTED. JOHAB is not supported by glibc any more
since 2000-09-25. TCVN5712-1 is not supported by glibc any more since
2012-05-21.
Bruno Haible [Thu, 13 Jul 2023 09:15:04 +0000 (11:15 +0200)]
c32_get_mapping: New module.
* lib/uchar.in.h (c32_mapping_t): New type.
(c32_get_mapping): New declaration.
* lib/c32_get_mapping.c: New file, based on lib/wctrans-impl.h.
* m4/uchar_h.m4 (gl_UCHAR_H_REQUIRE_DEFAULTS): Initialize
GNULIB_C32_GET_MAPPING.
* modules/uchar (Makefile.am): Substitute GNULIB_C32_GET_MAPPING.
* modules/c32_get_mapping: New file.
Bruno Haible [Wed, 12 Jul 2023 20:44:52 +0000 (22:44 +0200)]
c32_get_type_test: New module.
* lib/uchar.in.h (c32_type_test_t): New type.
(c32_get_type_test): New declaration.
* lib/c32_get_type_test.c: New file, based on lib/wctype-impl.h.
* m4/uchar_h.m4 (gl_UCHAR_H_REQUIRE_DEFAULTS): Initialize
GNULIB_C32_GET_TYPE_TEST.
* modules/uchar (Makefile.am): Substitute GNULIB_C32_GET_TYPE_TEST.
* modules/c32_get_type_test: New file.
Gavin Smith [Wed, 12 Jul 2023 14:00:40 +0000 (16:00 +0200)]
gendocs: support chapter- and section-level split
* doc/gendocs_template: Add lines to mark parts of file to output
only when splitting HTML by node.
Remove obsolete comment about ranges of years in copyright notice
(as per Karl).
* build-aux/gendocs.sh
[!texi2html]: Strip out parts of the template depending on --split.
[texi2html] Include all of the template as before.
Paul Eggert [Tue, 11 Jul 2023 14:54:01 +0000 (07:54 -0700)]
More than one initial mbstate_t
In commentary, say "an initial state" rather than
"the initial state" for mbstate_t, as it is possible
and indeed common for there to be more than one
initial state. POSIX routinely says "an initial state".
Bruno Haible [Mon, 10 Jul 2023 22:52:23 +0000 (00:52 +0200)]
dfa: Overcome wchar_t limitations.
* lib/localeinfo.h: Include <uchar.h>. Add special definitions for GAWK.
(case_folded_counterparts): Change array element type to char32_t.
* lib/localeinfo.c: Include <uchar.h>. Add special definitions for GAWK.
(is_using_utf8, init_localeinfo): Use mbrtoc32 instead of mbrtowc.
(lonesome_lower): Change element type to 'unsigned short'.
(case_folded_counterparts): Change array element type to char32_t. Use
c32toupper instead of towupper. Use c32tolower instead of towlower.
* lib/dfa.c: Include <uchar.h>. Add special definitions for GAWK.
(struct mb_char_classes): Change element type of 'chars' to char32_t.
(mbs_to_wchar): Use mbrtoc32 instead of mbrtowc.
(setbit_wc): Change type of first argument to char32_t. Use c32tob
instead of wctob.
(parse_bracket_exp): Update.
(lex): Use c32isprint instead of iswprint. Use c32isspace instead of
iswspace. Use c32rtomb instead of a %lc directive.
(addtok_wc): Use c32rtomb instead of wcrtomb.
(atom): Update.
* modules/dfa (Depends-on): Remove wctype-h. Add uchar,
mbrtoc32-regular, c32rtomb, c32tob, c32tolower, c32toupper, c32isprint,
c32isspace.
(Link): Add $(LIBUNISTRING) $(LIBC32CONV).
* modules/dfa-tests (Makefile.am): Link test-dfa-match-aux with
$(LIBUNISTRING) $(LIBC32CONV).
* NEWS: Mention the change.
Bruno Haible [Mon, 10 Jul 2023 22:03:34 +0000 (00:03 +0200)]
mbrtoc32-regular: New module.
* modules/mbrtoc32-regular: New file.
* lib/mbrtoc32.c (mbrtoc32): If the module 'mbrtoc32-regular' is present
and the system's mbrtoc32 returned a char32_t, clear the mbstate_t.
* doc/posix-functions/mbrtoc32.texi: Mention the new module.
Bruno Haible [Fri, 7 Jul 2023 10:04:14 +0000 (12:04 +0200)]
dfa: Overcome wchar_t limitations.
* lib/localeinfo.h: Include <uchar.h>. Add special definitions for GAWK.
(case_folded_counterparts): Change array element type to char32_t.
* lib/localeinfo.c: Include <uchar.h>. Add special definitions for GAWK.
(is_using_utf8, init_localeinfo): Use mbrtoc32 instead of mbrtowc.
(lonesome_lower): Change element type to 'unsigned short'.
(case_folded_counterparts): Change array element type to char32_t. Use
c32toupper instead of towupper. Use c32tolower instead of towlower.
* lib/dfa.c: Include <uchar.h>. Add special definitions for GAWK.
(struct mb_char_classes): Change element type of 'chars' to char32_t.
(mbs_to_wchar): Use mbrtoc32 instead of mbrtowc.
(setbit_wc): Change type of first argument to char32_t. Use c32tob
instead of wctob.
(parse_bracket_exp): Update.
(lex): Use c32isprint instead of iswprint. Use c32isspace instead of
iswspace. Use c32rtomb instead of a %lc directive.
(addtok_wc): Use c32rtomb instead of wcrtomb.
(atom): Update.
* modules/dfa (Depends-on): Remove wctype-h. Add uchar, mbrtoc32,
c32rtomb, c32tob, c32tolower, c32toupper, c32isprint, c32isspace.
(Link): Add $(LIBUNISTRING) $(LIBC32CONV).
* modules/dfa-tests (Makefile.am): Link test-dfa-match-aux with
$(LIBUNISTRING) $(LIBC32CONV).
* NEWS: Mention the change.
Paul Eggert [Sun, 9 Jul 2023 18:23:58 +0000 (11:23 -0700)]
c32width: protect against bad configure args
Avoid ‘test’ misuse with ‘./configure gt_cv_locale_fr=""’.
Possible problem of this sort reported by Angelo Graziosi in:
https://lists.gnu.org/r/emacs-devel/2023-07/msg00329.html
* m4/locale-fr.m4 (gt_LOCALE_FR_UTF8, gt_LOCALE_FR_UTF8):
* m4/locale-ja.m4 (gt_LOCALE_JA):
* m4/locale-zh.m4 (gt_LOCALE_ZH_CN):
Reject locale names that might cause trouble with sh or make.
Bruno Haible [Tue, 4 Jul 2023 19:04:00 +0000 (21:04 +0200)]
mbiter, mbfile, mbmemcasecoll: Improve handling of mbrtoc32 result.
* lib/mbiter.h (mbi_avail): If cur.ptr has reached the limit but
in_shift is true, call mbiter_multi_next.
(mbiter_multi_next): Set in_shift to false after an incomplete multibyte
character.
* lib/mbfile.h (mbfile_multi_getc): Pass the input bytes incrementally
into mbrtoc32. When mbf->state is not in the initial state, call
mbrtoc32 again.
* lib/mbmemcasecoll.c (apply_c32tolower): When the state is not in the
initial state, call mbrtoc32 again.
Bruno Haible [Tue, 4 Jul 2023 13:31:09 +0000 (15:31 +0200)]
mbchar: Optimize all chars from the POSIX "portable character set".
* lib/mbchar.h (is_basic): Include all the characters from the POSIX
"portable character set".
* lib/mbchar.c (is_basic_table): Likewise.
* lib/mbiter.h (mbiter_multi_next): Update comment.
* lib/mbuiter.h (mbuiter_multi_next): Likewise.
* lib/mbfile.h (mbfile_multi_getc): Likewise.
Paul Eggert [Mon, 3 Jul 2023 17:54:36 +0000 (10:54 -0700)]
mbchar: treat @, $, ` as basic
The C standard says that @, $, ` must have a single-byte
representation so they count as basic as far as multibyte
character processing goes.
* lib/mbchar.c (is_basic_table):
* lib/mbchar.h (is_basic):
* lib/mbswidth.c (mbsnwidth):
Treat @, $, ` as basic too.
Bruno Haible [Sun, 2 Jul 2023 14:46:11 +0000 (16:46 +0200)]
mbsrtoc32s: Accommodate a difference between mbrtowc and mbrtoc32.
* lib/mbsrtoc32s.c (USES_C32): New macro.
* lib/mbsrtowcs.c (USES_C32): New macro.
* lib/mbsrtowcs-impl.h (FUNC): If USES_C32 and mbrtoc32 returned
(size_t)(-3), don't increment the pointer.
Bruno Haible [Sun, 2 Jul 2023 14:45:44 +0000 (16:45 +0200)]
mbsnrtoc32s: Accommodate a difference between mbrtowc and mbrtoc32.
* lib/mbsnrtoc32s.c (USES_C32): New macro.
* lib/mbsnrtowcs.c (USES_C32): New macro.
* lib/mbsnrtowcs-impl.h (FUNC): If USES_C32 and mbrtoc32 returned
(size_t)(-3), don't increment the pointer.