Bruno Haible [Sun, 17 Jun 2018 21:12:54 +0000 (23:12 +0200)]
gettext po infrastructure: Update from current gettext git.
Reported by Akim Demaille <akim@lrde.epita.fr>.
* build-aux/po/Makefile.in.in: Update from current gettext git.
* build-aux/po/remove-potcdate.sin: Likewise.
* config/srclist.txt: Temporarily disable sync for these files.
Paul Eggert [Mon, 4 Jun 2018 01:02:10 +0000 (18:02 -0700)]
Port crypto/af_alg to GCC 4.8.4
Problem reported by Peter Simons in:
https://lists.gnu.org/r/bug-gnulib/2018-06/msg00002.html
* modules/crypto/af_alg (Depends-on): Add c99 if USE_AF_ALG.
Colin Watson [Sun, 27 May 2018 20:07:58 +0000 (22:07 +0200)]
bootstrap: allow non-submodule control of gnulib
Some projects prefer to avoid git submodules for various reasons (they
have rather strange UI), but at present they're the only way to pin a
particular gnulib commit. Add a couple of extra variables which can be
set in bootstrap.conf to control this separately.
* build-aux/bootstrap: Honour GNULIB_URL and GNULIB_REVISION in
bootstrap.conf when fetching gnulib using "git clone" or via
GNULIB_SRCDIR.
Paul Eggert [Mon, 21 May 2018 17:32:25 +0000 (10:32 -0700)]
crypto: omit stream ops Emacs doesn’t need
* lib/md5.c (md5_stream):
* lib/sha1.c (sha1_stream):
* lib/sha256.c (shaxxx_stream, sha256_stream, sha224_stream):
* lib/sha512.c (shaxxx_stream, sha512_stream, sha384_stream):
Compile stream functions only if GL_COMPILE_CRYPTO_STREAM is
defined. Emacs needs this, as it does not use the stream
operations and doesn’t need all the af_alg stuff we’ve recently
added. Perhaps a similar change is needed to the other crypto
modules, but this patch changes only those needed for Emacs.
* modules/crypto/md5-buffer, modules/crypto/sha1-buffer:
* modules/crypto/sha256-buffer, modules/crypto/sha512-buffer:
New modules, used by Emacs.
* modules/crypto/md5, modules/crypto/sha1, modules/crypto/sha256:
* modules/crypto/sha512: Rewrite to depend on the new modules.
Pádraig Brady [Mon, 14 May 2018 01:42:37 +0000 (18:42 -0700)]
fts: avoid a memory leak edge case
* lib/fts.c (fts_open): Set an appropriate fts_level
so that an immediate fts_close() will free the allocation.
* tests/test-fts.c (fts_dealloc): Add a test case which
will trigger under valgrind or address sanitizer.
Fixes https://bugs.gnu.org/31439
Bruno Haible [Sun, 20 May 2018 10:59:01 +0000 (12:59 +0200)]
localcharset: Optimize.
* lib/localcharset.c (alias_table): Comment out no-op mappings for
platforms where these don't matter. This reduces the table size,
which in turn reduces the lookup time.
Bruno Haible [Sun, 13 May 2018 23:48:22 +0000 (01:48 +0200)]
truncate: Fix compilation error on Android.
* m4/truncate.m4 (gl_FUNC_TRUNCATE): Test also whether 'truncate' is
declared. Set HAVE_DECL_TRUNCATE, not HAVE_TRUNCATE.
* lib/unistd.in.h (truncate): Test HAVE_DECL_TRUNCATE, not
HAVE_TRUNCATE.
* modules/truncate: Likewise.
* m4/unistd_h.m4 (gl_UNISTD_H_DEFAULTS): Initialize HAVE_DECL_TRUNCATE,
not HAVE_TRUNCATE.
* modules/unistd (Makefile.am): Substitute HAVE_DECL_TRUNCATE, not
HAVE_TRUNCATE.
* doc/posix-functions/truncate.texi: Mention the issue.
Bruno Haible [Sun, 13 May 2018 22:27:43 +0000 (00:27 +0200)]
posix_spawn: Fix compilation error on Android.
* lib/spawn.in.h (posix_spawnattr_t): Consider also the case
HAVE_POSIX_SPAWNATTR_T = 1 && HAVE_POSIX_SPAWN = 0.
(posix_spawn_file_actions_t): Consider also the case
HAVE_POSIX_SPAWN_FILE_ACTIONS_T = 1 && HAVE_POSIX_SPAWN = 0.
Bruno Haible [Sun, 13 May 2018 22:02:48 +0000 (00:02 +0200)]
tsearch: Fix compilation error on Android.
* lib/search.in.h (twalk): Declare when HAVE_TWALK, not HAVE_TSEARCH,
is 0.
(GNULIB_defined_tsearch, GNULIB_defined_twalk): New macros.
* lib/tsearch.c (tsearch, tfind, tdelete): Define only if
GNULIB_defined_tsearch is true.
(twalk): Define only if GNULIB_defined_twalk is true.
* modules/tsearch (configure.ac): Compile tsearch.c also if HAVE_TWALK
is 0.
* m4/tsearch.m4 (gl_FUNC_TSEARCH): Set HAVE_TWALK.
* m4/search_h.m4 (gl_SEARCH_H_DEFAULTS): Initialize HAVE_TWALK.
* modules/search (Makefile.am): Substitute HAVE_TWALK.
Bruno Haible [Sun, 13 May 2018 21:23:47 +0000 (23:23 +0200)]
imaxdiv: Fix compilation error on Android.
* m4/imaxdiv.m4 (gl_FUNC_IMAXDIV): Set HAVE_IMAXDIV_T to 0 if imaxdiv_t
is not defined.
* lib/inttypes.in.h (imaxdiv_t): Define if HAVE_IMAXDIV_T, not
HAVE_DECL_IMAXDIV, is 0.
* m4/inttypes.m4 (gl_INTTYPES_H_DEFAULTS): Initialize HAVE_IMAXDIV_T.
* modules/inttypes-incomplete (Makefile.am): Substitute HAVE_IMAXDIV_T.
Bruno Haible [Sun, 13 May 2018 20:20:13 +0000 (22:20 +0200)]
Support selective inclusion mechanism of recent mingw.org header files.
Reported by Eli Zaretskii <eliz@gnu.org>.
* lib/sys_types.in.h: On mingw, when __need_off_t, __need___off64_t,
__need_ssize_t, or __need_time_t is defined, just include the system's
<sys/types.h>.
* lib/locale.in.h: On mingw, when __need_locale_t is defined, just
include the system's <locale.h>.
Paul Eggert [Thu, 10 May 2018 01:07:29 +0000 (18:07 -0700)]
af_alg: recover better from crypto failures
* lib/af_alg.c (afalg_stream): Recover from crypto failures if the
input stream is seekable, by repositioning the stream back to
where it was, possibly by just calling sendfile with an offset
arg. This lets us return -EAFNOSUPPORT instead of -EIO in some
cases, which lets our callers try again with user-mode code.
* modules/crypto/af_alg (Depends-on): Depend on fseeko and ftello
instead of on fflush and lseek.
Paul Eggert [Wed, 9 May 2018 19:04:37 +0000 (12:04 -0700)]
af_alg: distiguish I/O errors better
* lib/af_alg.c (afalg_buffer, afalg_stream): Return -EAFNOSUPPORT,
not -EIO, if it’s OK for the caller to try again with user-mode code.
(afalg_stream) [!_WIN32 || __CYGWIN__]: Return -EIO (not possibly
some other error number) if fflush fails, as the caller should not
try again that case.
Paul Eggert [Wed, 9 May 2018 18:34:28 +0000 (11:34 -0700)]
af_alg: don’t leak file descriptors into children
* lib/af_alg.c (alg_socket): Use SOCK_CLOEXEC when creating sockets.
This code should be compiled only on recent GNU/Linux platforms
so we shouldn’t have to also depend on the accept4 module.
Paul Eggert [Mon, 7 May 2018 07:25:57 +0000 (00:25 -0700)]
af_alg: Pacify --enable-gcc-warnings
Problem reported by Assaf Gordon in:
https://lists.gnu.org/r/bug-gnulib/2018-05/msg00041.html
* lib/af_alg.c (afalg_buffer): Move local decls to pacify
gcc -Wjump-misses-init.
* lib/sha512.c (shaxxx_stream): Now static.
Paul Eggert [Sun, 6 May 2018 02:39:37 +0000 (19:39 -0700)]
crypto/{md5,sha1,sha256,sha512}: simplify
* lib/md5.c (md5_stream):
* lib/sha1.c (sha1_stream):
* lib/sha256.c (shaxxx_stream):
Simplify, partly by assuming C99.
* lib/sha256.c (shaxxx_stream):
New function, which implements both sha256 and sha224.
Simplify, partly by assuming C99.
(sha256_stream, sha224_stream):
Use it to avoid code duplication, removing a FIXME.
* lib/sha512.c (shaxxx_stream, sha512_stream, sha384_stream):
Likewise.
Bruno Haible [Sat, 5 May 2018 18:27:33 +0000 (20:27 +0200)]
af_alg: New module.
* lib/af_alg.h: Test HAVE_* macro through '#if', not '#ifdef'.
* lib/af_alg.c: Include "af_alg.h" before the other header files.
* lib/md5.c: Include "af_alg.h" unconditionally.
(md5_stream): Invoke afalg_stream unconditionally.
* lib/sha1.c: Include "af_alg.h" unconditionally.
(sha1_stream): Invoke afalg_stream unconditionally.
* lib/sha256.c: Include "af_alg.h" unconditionally.
(sha256_stream, sha224_stream): Invoke afalg_stream unconditionally.
* lib/sha512.c: Include "af_alg.h" unconditionally.
(sha512_stream, sha384_stream): Invoke afalg_stream unconditionally.
* m4/af_alg.m4: Renamed from m4/linux-if-alg.m4.
(gl_AF_ALG): Renamed from gl_LINUX_IF_ALG_H.
* modules/crypto/af_alg: New file.
* modules/crypto/md5 (Files): Remove files that are now in the
'crypto/af_alg' module.
(Depends-on): Add crypto/af_alg.
(configure.ac): Remove gl_LINUX_IF_ALG_H invocation.
(Makefile.am): Don't mention af_alg.c here.
* modules/crypto/sha1 (Files): Remove files that are now in the
'crypto/af_alg' module.
(Depends-on): Add crypto/af_alg.
(configure.ac): Remove gl_LINUX_IF_ALG_H invocation.
(Makefile.am): Don't mention af_alg.c here.
* modules/crypto/sha256 (Files): Remove files that are now in the
'crypto/af_alg' module.
(Depends-on): Add crypto/af_alg.
(configure.ac): Remove gl_LINUX_IF_ALG_H invocation.
(Makefile.am): Don't mention af_alg.c here.
* modules/crypto/sha512 (Files): Remove files that are now in the
'crypto/af_alg' module.
(Depends-on): Add crypto/af_alg.
(configure.ac): Remove gl_LINUX_IF_ALG_H invocation.
(Makefile.am): Don't mention af_alg.c here.
Paul Eggert [Sat, 5 May 2018 18:08:08 +0000 (11:08 -0700)]
af_alg: minor style improvements
* lib/af_alg.c (afalg_stream): Prefer C99 style
decl-after-statement, since we’re already assuming C99. Clarify
by strengthening the bind test and omit unnecessary assignment.
Paul Eggert [Sat, 5 May 2018 17:27:38 +0000 (10:27 -0700)]
sys-limits.h: new file for crypto and safe I/O
* lib/af_alg.c: Include sys-limits.h.
(MAX_RW_COUNT): Remove. Use replaced by SYS_BUFSIZE_MAX.
(afalg_stream): Also reject negative sizes for sendfile; they
should not happen and the code is a bit cleaner and faster this way.
* lib/safe-read.c: Include sys-limits.h.
(BUGGY_READ_MAXIMUM): Remove. All uses replaced by SYS_BUFSIZE_MAX.
* lib/sys-limits.h: New file, with values and commentary derived
from the old safe-read.c and from GNU Emacs sysdep.c.
* modules/crypto/md5, modules/crypto/sha1, modules/crypto/sha256:
* modules/crypto/sha512, modules/safe-read, modules/safe-write:
Add lib/sys-limits.h to Files section.
Linux supports accessing kernel crypto API via AF_ALG since
version 2.6.38. Coreutils uses libcrypto when available and fallbacks to
generic C implementation of various hashing functions.
Add a generic afalg_stream() function which uses AF_ALG to calculate the
hash of a stream and use sendfile() when possible (regular file with size
less or equal than 0x7ffff000 (2,147,479,552) bytes, AKA MAX_RW_COUNT).
Use afalg_stream() only in sha1sum for now, but other hashes are possible.
The speed gain really depends on the CPU type, on systems which doesn't use
libcrypto ranges from ~10% to 320%.
This is a test on a Intel(R) Xeon(R) CPU E3-1265L V2 and Debian stretch: