Paul Eggert [Thu, 17 Dec 2020 09:22:55 +0000 (01:22 -0800)]
canonicalize-lgpl: fix AIX test failures
This merges the recent canonicalize.c fix into canonicalize-lgpl.c.
Problem reported by Bruno Haible in:
https://lists.gnu.org/r/bug-gnulib/2020-12/msg00138.html
* lib/canonicalize-lgpl.c: Include sys/stat.h.
(__realpath): When testing a file name ending in '/', use stat
rather than readlink, so that it does the right thing on AIX.
* modules/canonicalize-lgpl (Depends-on): Add stat, sys_stat.
Paul Eggert [Thu, 17 Dec 2020 07:50:34 +0000 (23:50 -0800)]
canonicalize: remove arbitrary 8192-byte limit
Remove canonicalize.c’s arbitrary 8192-byte limit on platforms
like GNU Hurd that do not define the PATH_MAX macro, and similarly
for canonicalize-lgpl.c’s arbitrary 1024-byte limit. Do this by
using scratch buffers. Lessen the number of differences between
the two source files, to simplify this and future maintenance.
* lib/canonicalize-lgpl.c (__realpath):
* lib/canonicalize.c (canonicalize_filename_mode_stk):
Use scratch buffers instead of malloc and malloca. This avoids
the need for alloca, and avoids the need for malloc in most cases.
* lib/canonicalize-lgpl.c, lib/canonicalize.c: Make these files
easier to compare, e.g., by sorting include files and by switching
to the GNU convention of calling file names "file names", not
"path names". Include stdbool.h, scratch_buffer.h.
* lib/canonicalize-lgpl.c (IDX_MAX) [_LIBC]: New macro.
(malloca) [_LIBC]: Remove.
[!_LIBC]: Do not include malloca.h.
(get_path_max): New function, so that pathconf is called only in
the rare and dubious case when when RESOLVED is not null and
PATH_MAX is not defined. Invoke pathconf on "/" not the input
file name, as we care about the longest file name starting from
"/" (not from the input file name), and POSIX does not specify
what pathconf does on a non-directory file anyway. If PATH_MAX is
not defined, do not worry about overriding a path_max of 0, and do
not let path_max exceed IDX_MAX.
(__realpath): Remove an assumption that file name components
cannot exceed 1024 bytes when PATH_MAX is not defined (wrong for
the Hurd, presumably).
When allocating the result, allocate it to just the right size;
this costs nothing when the result is smaller than 1023 bytes,
and for larger results it's probably worth the CPU to call realloc,
as canonicalize.c already does.
* lib/canonicalize.c: Include attribute.h.
Do not include pathmax.h or xgetcwd.h.
(PATH_MAX): Do not define, so file names longer than 8192 bytes
work on platforms with no fixed limit.
(canonicalize_filename_mode_stk): New function, with
the content of the old canonicalize_filename_mode.
Use getcwd instead of xgetcwd, and readlink instead of areadlink,
since the scratch buffers now do memory management for us.
Use rawmemchr instead of adding strlen.
Use mempcpy instead of mempcpy + size.
Assume free preserves errno.
(canonicalize_filename_mode): Use it.
* modules/canonicalize (Depends-on): Remove areadlink, pathmax,
xgetcwd. Add attribute, free, getcwd, mempcpy, rawmemchr,
scratch_buffer, stdbool, xalloc-die.
* modules/canonicalize-lgpl (Depends-on): Remove alloca-opt,
malloca, realloc-posix. Add scratch_buffer, stdbool.
Paul Eggert [Mon, 14 Dec 2020 19:59:20 +0000 (11:59 -0800)]
canonicalize-lgpl: simplify merge to glibc
This patch lessens the differences between git glibc
stdlib/canonicalize.c and lib/canonicalize-lgpl.c.
The (perhaps wishful) goal is to make them identical.
* lib/canonicalize-lgpl.c [!_LIBC]:
Include <libc-config.h>, not config.h.
Omit an unnecessary (!HAVE_CANONICALIZE_FILE_NAME ||
!FUNC_REALPATH_WORKS || defined _LIBC) #if.
Do not include alloca.h, since we use malloca now.
[_LIBC]: Include <eloop-threshold.h>, and define dummy macros
FILE_SYSTEM_PREFIX_LEN, IS_ABSOLUTE_FILE_NAME, ISSLASH, malloca,
freea so that the mainline code can be kept #ifdef free.
[!_LIBC]: Remove dummy macros for SHLIB_COMPAT, versioned_symbol,
compat_symbol, weak_alias, __set_errno since libc-config.h does that.
Add redirecting macros __mempcpy, __pathconf, __rawmemchr,
__eloop_threshold. All uses of their definiens changed.
(SIZE_MAX): Remove; no longer needed.
(alloc_failed): Remove, and remove all instances.
No need for alloc_failed now that free preserves errno.
(__realpath): Default path_max to 1024 instead of 8192, as that’s
the glibc tradition and is safer when the 2nd argument is null.
Use __rawmemchr instead of strchr.
Use __mempcpy where appropriate.
Simplify test for overflow so that it does not need SIZE_MAX.
Do not preserve errno around free or freea calls; no longer needed.
Mark __realpath with libc_hidden_def.
* modules/canonicalize-lgpl (Depends-on): Add free, libc-config,
malloc-posix, mempcpy, realloc-posix, rawmemchr.
* modules/free: Now LGPLv2+, for canonicalize-lgpl.
Paul Eggert [Mon, 14 Dec 2020 19:38:18 +0000 (11:38 -0800)]
free: preserve errno
* lib/free.c (rpl_free): Preserve errno. Check for null only if
CANNOT_FREE_NULL is defined, as an optimization for POSIX 2008
platforms that do not preserve errno.
* m4/free.m4 (gl_FUNC_FREE): Check whether free preserves errno.
Also, define CANNOT_FREE_NULL if free cannot free NULL.
* modules/free (configure.ac): Also replace 'free' if
it does not preserve errno.
Paul Eggert [Thu, 17 Dec 2020 07:52:24 +0000 (23:52 -0800)]
idx: simplify IDX_MAX, remove IDX_WIDTH
* lib/idx.h (IDX_MAX): Simplify by removing obsolete reference
to UNSIGNED_IDX_T.
(IDX_WIDTH): Remove, since it’s not used and its value
arguably should be PTRDIFF_WIDTH anyway.
Bruno Haible [Mon, 14 Dec 2020 18:22:14 +0000 (19:22 +0100)]
posix_spawn-internal: Make better use of 'const'.
* lib/spawn_int.h (__spawni): Does not need write access to the elements
of argv and envp.
* lib/spawni.c (__spawni, script_execute): Likewise.
* lib/spawn.c (posix_spawn): Update caller.
* lib/spawnp.c (posix_spawnp): Likewise.
Paul Eggert [Mon, 14 Dec 2020 01:16:01 +0000 (17:16 -0800)]
canonicalize: fix AIX test failures
Problem reported by Bruno Haible in:
https://lists.gnu.org/r/bug-gnulib/2020-12/msg00109.html
* lib/canonicalize.c (canonicalize_filename_mode):
When testing a file name ending in '/', use stat rather than
readlink, so that it does the right thing on AIX.
* modules/canonicalize (Depends-on): Add readlink, to pull in the
recent changes in the Gnulib readlink module.
Paul Eggert [Mon, 14 Dec 2020 00:56:10 +0000 (16:56 -0800)]
Assume readlink/readlinkat ERANGE fix
* lib/areadlink-with-size.c (areadlink_with_size):
* lib/areadlinkat-with-size.c (areadlinkat_with_size):
* lib/careadlinkat.c (readlink_stk):
Do not worry about readlink or readlinkat failing with errno == ERANGE,
since the Gnulib readlink and readlinkat modules now fix that.
Paul Eggert [Mon, 14 Dec 2020 00:48:26 +0000 (16:48 -0800)]
readlink, readlinkat: add ERANGE portability
Fix some portability issues with Gnulib's readlink and readlinkat,
notably mostly working around the ERANGE problem in AIX and HP-UX.
* doc/posix-functions/readlink.texi:
* doc/posix-functions/readlinkat.texi:
ERANGE problem is mostly fixed now. Mention AIX problem with
trailing / and EINVAL. Lessen differences between these two files.
* lib/readlink.c (rpl_readlink):
* lib/readlinkat.c (rpl_readlinkat):
If stat ("FILE/", ...) reports EOVERFLOW, treat FILE/ as an
existing directory. Mostly work around READLINK_TRUNCATE BUG.
Lessen spurious differences between the readlink and readlinkat code.
* lib/readlinkat.c (rpl_readlinkat):
Fix bug where stat was used where fstatat was intended.
* m4/readlink.m4 (gl_FUNC_READLINK):
Rename gl_cv_func_readlink_works to gl_cv_func_readlink_trailing_slash
to identify readlink problems more precisely. All uses changed.
Guess no on AIX or HP-UX for this variable.
Add check for whether readlink truncates results,
and define new macro READLINK_TRUCATE_BUG accordingly.
* m4/readlinkat.m4 (gl_FUNC_READLINKAT):
Also check gl_cv_func_readlink_trailing_slash when deciding
whether to replace readlinkat.
* modules/readlinkat (Depends-on): Most dependencies are also
needed if replacing readlinkat. fstatat is different, as it
is needed only if replacing an existing readlinkat.
Bruno Haible [Sat, 12 Dec 2020 03:18:54 +0000 (04:18 +0100)]
sh-quote, execute, spawn-pipe, etc.: Make better use of 'const'.
* lib/sh-quote.h (shell_quote_argv): Does not need write access to the
elements of argv.
* lib/sh-quote.c (shell_quote_argv): Likewise.
* lib/windows-spawn.h (prepare_spawn): Add 'const' the argument type and
the return type.
* lib/windows-spawn.c (prepare_spawn): Likewise.
* lib/os2-spawn.h (prepare_spawn): Likewise.
* lib/os2-spawn.c (prepare_spawn): Likewise.
* lib/execute.h (execute): Does not need write access to the elements of
prog_argv.
* lib/execute.c (execute): Likewise.
* lib/spawn-pipe.h (create_pipe_out, create_pipe_in, create_pipe_bidi):
Likewise.
* lib/spawn-pipe.c (create_pipe, create_pipe_bidi, create_pipe_in,
create_pipe_out): Likewise.
* lib/pipe-filter.h (pipe_filter_ii_execute, pipe_filter_gi_create):
Likewise.
* lib/pipe-filter-ii.c (pipe_filter_ii_execute): Likewise.
* lib/pipe-filter-gi.c (pipe_filter_gi_create): Likewise.
* lib/javaexec.h (execute_fn): Does not need write access to the
elements of prog_argv.
* lib/javaexec.c (execute_java_class): Update variable types and remove
casts to 'char *'.
* lib/csharpexec.h (execute_fn): Does not need write access to the
elements of prog_argv.
* lib/csharpexec.c (execute_csharp_using_mono,
execute_csharp_using_sscli): Update variable types and remove casts to
'char *'.
* lib/javacomp.c (compile_using_envjavac, compile_using_gcj,
compile_using_javac, compile_using_jikes, is_envjavac_gcj,
is_envjavac_gcj43, is_gcj_present, is_gcj_43, is_javac_present,
is_jikes_present): Update variable types and remove casts to 'char *'.
* lib/javaversion.c (execute_and_read_line): Does not need write access
to the elements of prog_argv.
* lib/csharpcomp.c (compile_csharp_using_mono,
compile_csharp_using_sscli): Update variable types and remove casts to
'char *'.
* tests/test-sh-quote.c (main): Update variable types and remove casts
to 'char *'.
* tests/test-execute-main.c (main): Update variable types and remove
casts to 'char *'.
* tests/test-spawn-pipe-main.c (test_pipe): Update variable types and
remove casts to 'char *'.
* NEWS: Mention the changes.
Paul Eggert [Sat, 12 Dec 2020 01:33:36 +0000 (17:33 -0800)]
stdint: port to Autoconf 2.70
* m4/stdint.m4 (gl_STDINT_H): Check for inttypes.h and sys/types.h
instead of assuming that AC_INCLUDES_DEFAULT does it.
The old code relied on AC_INCLUDES_DEFAULT being called
and setting ac_cv_header_inttypes_h and ac_cv_header_sys_types_h,
but this does not occur in Autoconf 2.70.
Paul Eggert [Fri, 11 Dec 2020 23:50:34 +0000 (15:50 -0800)]
AC_C_RESTRICT: update from Autoconf
* m4/gnulib-common.m4 (gl_PROG_CC_C99): Use Autoconf 2.70
as a prerequisite, not 2.69c, since 2.70 is now out.
(AC_C_RESTRICT): Define only for 2.70 or earlier.
Try __restrict__ before __restrict.
Paul Eggert [Fri, 11 Dec 2020 23:47:24 +0000 (15:47 -0800)]
extensions: update from Autoconf
* m4/extensions.m4 (AC_CHECK_INCLUDES_DEFAULT):
Provide a default implementation for Autoconf 2.69 or earlier.
(AC_USE_SYSTEM_EXTENSIONS): Copy from Autoconf git. Define only
if Autoconf 2.70 or earlier, since 2.70.1 or later should be OK.
Bruno Haible [Fri, 11 Dec 2020 10:47:30 +0000 (11:47 +0100)]
execute-tests: Fix compilation error on AIX in 32-bit mode.
* tests/test-execute-child.c: In order to get the original definition of
fstat, don't use '#undef fstat' and '#undef stat'. Instead, arrange to
include the system's <sys/stat.h> and use it before including other
header files.
Bruno Haible [Thu, 10 Dec 2020 22:43:20 +0000 (23:43 +0100)]
execute, spawn-pipe: Fix memory leak on native Windows.
* lib/windows-spawn.h (prepare_spawn): Add a second parameter.
* lib/windows-spawn.c: Don't include xalloc.h.
(quoted_arg_length, quoted_arg_string): New functions, extracted from
prepare_spawn.
(prepare_spawn): Use malloc instead of XNMALLOC. Allocate memory for all
elements of *new_argv together.
* modules/windows-spawn (Depends-on): Remove xalloc. Add malloc-posix.
* lib/os2-spawn.h (prepare_spawn): Add a second parameter.
* lib/os2-spawn.c: Don't include xalloc.h.
(prepare_spawn): Use malloc instead of XNMALLOC. Allocate memory for all
elements of *new_argv together.
* lib/execute.c: Include xalloc.h.
(execute): Check return value of prepare_spawn. Free the memory
allocated by prepare_spawn.
* modules/execute (Depends-on): Add xalloc-die.
* lib/spawn-pipe.c: Include xalloc.h.
(create_pipe): Check return value of prepare_spawn. Free the memory
allocated by prepare_spawn.
* modules/spawn-pipe (Depends-on): Add xalloc-die.
Bruno Haible [Wed, 9 Dec 2020 17:51:40 +0000 (18:51 +0100)]
threadlib: Fix test-fstrcmp failure on FreeBSD 11.
* m4/threadlib.m4 (gl_THREADLIB_BODY): When weak symbols are not present
on FreeBSD, define PTHREAD_IN_USE_DETECTION_HARD.
* lib/glthread/threadlib.c: Include <errno.h>.
(glthread_in_use): For FreeBSD, provide an alternative implementation
that uses pthread_key_create.
Bruno Haible [Tue, 8 Dec 2020 20:28:18 +0000 (21:28 +0100)]
argp: Avoid undefined behaviour when invoking qsort().
This fixes a test-argp-2.sh test failure on macOS and FreeBSD.
Reported by Jeffrey Walton <noloader@gmail.com> in
<https://lists.gnu.org/archive/html/bug-gnulib/2020-03/msg00085.html>.
* lib/argp-help.c (group_cmp): Remove third argument.
(hol_sibling_cluster_cmp, hol_cousin_cluster_cmp): New functions, based
upon hol_cluster_cmp.
(hol_cluster_cmp): Use hol_cousin_cluster_cmp.
(hol_entry_cmp): Rewritten to implement a total order.
Bruno Haible [Mon, 7 Dec 2020 18:43:06 +0000 (19:43 +0100)]
get-rusage-data: Fix link error on FreeBSD 12.2/arm64.
* modules/get-rusage-data (configure.ac): Test whether sbrk exists.
* lib/get-rusage-data.c (get_rusage_data_via_setrlimit): Define
trivially of sbrk is not available.
* doc/glibc-functions/sbrk.texi: Mention that the function does not
exist in FreeBSD 12.2/arm64.
* doc/glibc-functions/brk.texi: Likewise.
Paul Eggert [Sun, 6 Dec 2020 18:17:39 +0000 (10:17 -0800)]
doc: fix flat address space discussion
* doc/gnulib-readme.texi (Other portability assumptions):
Move the all-bits-zero assumption outside the flat address space
section, since the two issues are independent.
Bruno Haible [Sun, 6 Dec 2020 14:51:41 +0000 (15:51 +0100)]
doc: Mention some missing function declarations.
* doc/glibc-functions/execvpe.texi: Mention the missing declaration on
AIX.
* doc/glibc-functions/fcloseall.texi: Mention the missing declaration on
FreeBSD.
* doc/pastposix-functions/ecvt.texi: Mention the missing declaration on
Cygwin.
* doc/pastposix-functions/fcvt.texi: Likewise.
* doc/pastposix-functions/gcvt.texi: Likewise.
Bruno Haible [Sun, 6 Dec 2020 11:31:11 +0000 (12:31 +0100)]
parse-datetime: Use idx_t for nonnegative ptrdiff_t variables.
* lib/parse-datetime.y: Include idx.h.
(textint): Mark digits as nonnegative.
(parser_control): Mark dates_seen, days_seen, local_zones_seen,
dsts_seen, times_seen, zones_seen as nonnegative.
(lookup_word): Mark wordlen as nonnegative.
(yylex): Mark count as nonnegative.
(parse_datetime2): Mark tzsize as nonnegative.
* modules/parse-datetime (Depends-on): Add idx.
Paul Eggert [Sun, 6 Dec 2020 01:33:29 +0000 (17:33 -0800)]
doc: fix curved quotes issue
* doc/gnulib.texi: Set txicodequoteundirected and
txicodequotebacktick so that ` and ' in examples do not generate
curved single quotes that do the wrong thing when cut and pasted.
Paul Eggert [Sat, 5 Dec 2020 23:54:22 +0000 (15:54 -0800)]
intprops: Add INT_ADD_OK etc.
* doc/intprops.texi (Checking Integer Overflow): New section.
* lib/intprops.h: From a suggestion by Bruno Haible in:
https://lists.gnu.org/r/bug-gnulib/2020-12/msg00051.html
(SAFE_INT_ADD, SAFE_INT_SUBTRACT, SAFE_INT_MULTIPLY): New macros.
Paul Eggert [Sat, 5 Dec 2020 21:10:28 +0000 (13:10 -0800)]
doc: move exotic platfroms to Target Platforms
* doc/gnulib-intro.texi (Supported Platforms)
(Formerly Supported Platforms, Unsupported Platforms):
New subsections, split off from Target Platforms.
(Unsupported Platforms): Move the exotic-platform stuff here ...
* doc/gnulib-readme.texi (Exotic platforms): ... from this removed
section.
Paul Eggert [Sat, 5 Dec 2020 19:27:13 +0000 (11:27 -0800)]
doc: mention intptr_t etc. and IBM i
* doc/gnulib-readme.texi (Other portability assumptions):
Mention intptr_t and uintptr_t, and that arithmetic on them
works in the usual way.
(Exotic platforms): New section, containing material from
the old 'Integer Portability' section. Also mention IBM i.
* doc/intprops.texi (Wraparound Arithmetic):
Say that the macros work on unsigned integers too.
(Integer Portability): Remove.
Paul Eggert [Fri, 4 Dec 2020 18:44:51 +0000 (10:44 -0800)]
intprops: update doc and mention Unisys
* doc/gnulib-readme.texi (Other portability assumptions):
Also mention ptrdiff_t when talking about widths and overflow.
* doc/intprops.texi (Integer Properties): Summarize new section.
(Arithmetic Type Properties): Document that EXPR_SIGNED no longer
evaluates its argument.
(Integer Bounds): Fix typo.
(Wraparound Arithmetic): Remove obsolete comment about efficiency.
Document that the _WRAPV macros now support pointers to unsigned
integers.
(Integer Range Overflow): Update SEI CERT citation.
(Integer Portability): New subsection, which mentions
the oddball Unisys platforms as non-Gnulib targets.
Bruno Haible [Thu, 3 Dec 2020 20:00:35 +0000 (21:00 +0100)]
fprintf-posix-tests: Avoid a test failure on macOS 10.13.
Reported by Martin Storsjö <martin@martin.st> in
<https://lists.gnu.org/archive/html/bug-gnulib/2020-12/msg00003.html>.
* tests/test-fprintf-posix3.c: Skip the test on macOS.
(main): Return a different exit code at each point. Allow 100 KB extra
memory consumption.
* tests/test-fprintf-posix3.sh: Update. Remove the "get_rusage_as()
doesn't work" diagnostic.
Paul Eggert [Wed, 2 Dec 2020 22:25:43 +0000 (14:25 -0800)]
canonicalize: refactor can_mode flag
* lib/canonicalize.c (MULTIPLE_BITS_SET): Remove, replacing with ...
(multiple_bits_set): ... this new static function. Uses changed.
(canonicalize_filename_mode): Refactor for clarity to avoid
modifying the CAN_MODE argument.
Paul Eggert [Wed, 2 Dec 2020 22:25:43 +0000 (14:25 -0800)]
canonicalize: prefer signed integer types
* lib/canonicalize.c: Include stddef.h, for ptrdiff_t.
(seen_triple, canonicalize_filename_mode): Prefer signed to
unsigned types where either will do, as they avoid some glitches
in comparisons and can trap on overflow when debugging.
Paul Eggert [Wed, 2 Dec 2020 22:25:42 +0000 (14:25 -0800)]
canonicalize: fix most of another EOVERFLOW issue
* lib/canonicalize.c (canonicalize_filename_mode):
Do not call stat if fewer than 20 symlinks have been traversed.
This avoids EOVERFLOW failure in the common case where there
are not that many symlinks, while continuing to catch loops
(or fail due to EOVERFLOW) in the unusual case when there
are many symlinks to traverse.
Paul Eggert [Wed, 2 Dec 2020 22:25:42 +0000 (14:25 -0800)]
canonicalize: do not assume symlinks have st_ino
* lib/canonicalize.c (canonicalize_filename_mode):
When checking for loops, use st_dev and st_ino from the parent
directory not from the symlink, as pre-2017 POSIX says these
members are not reliable for symlinks. Couple this with START
(the remaining file name to be resolved), not NAME (the whole file
name with START as its suffix).
* modules/canonicalize (Depends-on): Depend on stat, not lstat.
Paul Eggert [Wed, 2 Dec 2020 22:25:41 +0000 (14:25 -0800)]
canonicalize: fix EOVERFLOW bug
* lib/canonicalize.c (canonicalize_filename_mode):
When testing whether a directory entry is a symbolic link, or a
directory or other, do not use lstat or stat or
areadlink_with_size. Just use areadlink, as this suffices and it
avoids the EOVERFLOW problem that lstat and stat have.
* modules/canonicalize (Depends-on): Depend on areadlink instead
of areadlink-with-size and stat.
Paul Eggert [Wed, 2 Dec 2020 22:25:41 +0000 (14:25 -0800)]
canonicalize-lgpl: fix EOVERFLOW bug
* lib/canonicalize-lgpl.c: Do not include <sys/stat.h>.
(__realpath): Do not use lstat. Just use readlink, as this
suffices and it avoids the EOVERFLOW problem that lstat has.
* modules/canonicalize-lgpl (Depends-on): Remove lstat, sys_stat.
Bruno Haible [Tue, 1 Dec 2020 18:57:17 +0000 (19:57 +0100)]
vma-iter: Add support for macOS11/arm64.
Patch suggested by Hill Ma <maahiuzeon@gmail.com> in
<https://gitlab.com/gnu-clisp/clisp/-/issues/27>
and by Martin Storsjö <martin@martin.st> in
<https://lists.gnu.org/archive/html/bug-gnulib/2020-12/msg00003.html>.
* lib/vma-iter.c (vma_iterate): On arm64, use 64-bit type definitions.