Commit Graph
100 Commits
Author SHA1 Message Date
Szabolcs NagyandRich Felker d20558148d math: fix sinh overflows in non-nearest rounding
The final rounding operation should be done with the correct sign
otherwise huge results may incorrectly get rounded to or away from
infinity in upward or downward rounding modes.

This affected sinh and sinhf which set the sign on the result after
a potentially overflowing mul. There may be other non-nearest rounding
issues, but this was a known long standing issue with large ulp error
(depending on how ulp is defined near infinity).

The fix should have no effect on sinh and sinhf performance but may
have a tiny effect on cosh and coshf.
2020-02-21 23:42:12 -05:00
Szabolcs NagyandRich Felker b3797d3b2e math: fix __rem_pio2 in non-nearest rounding modes
Handle when after reduction |y| > pi/4+tiny. This happens in directed
rounding modes because the fast round to int code does not give the
nearest integer. In such cases the reduction may not be symmetric
between x and -x so e.g. cos(x)==cos(-x) may not hold (but polynomial
evaluation is not symmetric either with directed rounding so fixing
that would require more changes with bigger performance impact).

The fix only adds two predictable branches in nearest rounding mode,
simple ubenchmark does not show relevant performance regression in
nearest rounding mode.

The code could be improved: e.g reducing the medium size threshold
such that two step reduction is enough instead of three, and the
single precision case can avoid the issue by doing the round to int
differently, but this fix was kept minimal.
2020-02-21 23:42:05 -05:00
Szabolcs NagyandRich Felker 2fc29fcfc2 mips: add clone3 syscall numbers from linux v5.4
the syscall numbers were reserved in v5.3 but not wired up on mips, see

  linux commit 0671c5b84e9e0a6d42d22da9b5d093787ac1c5f3
  MIPS: Wire up clone3 syscall
2019-12-30 18:14:08 -05:00
Szabolcs NagyandRich Felker a7df9c75f7 mips: add hwcap bits from linux v5.4
mips application specific isa extensions were previously not exported
in hwcaps so userspace could not apply optimized code at runtime.

  linux commit 38dffe1e4dde1d3174fdce09d67370412843ebb5
  MIPS: elf_hwcap: Export userspace ASEs
2019-12-30 18:14:08 -05:00
Szabolcs NagyandRich Felker f296be74d5 sys/wait.h: add P_PIDFD from linux v5.4
allows waiting on a pidfd, in the future it might allow retrieving the
exit status by a non-parent process, see

  linux commit 3695eae5fee0605f316fbaad0b9e3de791d7dfaf
  pidfd: add P_PIDFD to waitid()
2019-12-30 18:14:08 -05:00
Szabolcs NagyandRich Felker 5e0c9f246c netinet/tcp.h: add new tcp_info fields from linux v5.4
tcpi_rcv_ooopack for tracking connection quality:

  linux commit f9af2dbbfe01def62765a58af7fbc488351893c3
  tcp: Add TCP_INFO counter for packets received out-of-order

tcpi_snd_wnd peer window size for diagnosing tcp performance problems:

  linux commit 8f7baad7f03543451af27f5380fc816b008aa1f2
  tcp: Add snd_wnd to TCP_INFO
2019-12-30 18:14:08 -05:00
Szabolcs NagyandRich Felker 5675aaa559 sys/prctl.h: add PR_*_TAGGED_ADDR_* from linux v5.4
per thread prctl commands to relax the syscall abi such that top bits
of user pointers are ignored in the kernel. this allows the use of
those bits by hwasan or by mte to color pointers and memory on aarch64:

  linux commit 63f0c60379650d82250f22e4cf4137ef3dc4f43d
  arm64: Introduce prctl() options to control the tagged user addresses ABI
2019-12-30 18:14:08 -05:00
Szabolcs NagyandRich Felker d95ead261e sys/mman.h: add MADV_COLD and MADV_PAGEOUT from linux v5.4
These were mainly introduced so android can optimize the memory usage
of unused apps.

MADV_COLD hints that the memory range is currently not needed (unlike
with MADV_FREE the content is not garbage, it needs to be swapped):

  linux commit 9c276cc65a58faf98be8e56962745ec99ab87636
  mm: introduce MADV_COLD

MADV_PAGEOUT hints that the memory range is not needed for a long time
so it can be reclaimed immediately independently of memory pressure
(unlike with MADV_DONTNEED the content is not garbage):

  linux commit 1a4e58cce84ee88129d5d49c064bd2852b481357
  mm: introduce MADV_PAGEOUT
2019-12-30 18:14:08 -05:00
Szabolcs NagyandRich Felker fa765ca54f add clone3 syscall number from linux v5.3
the syscall number is reserved on all targets, but it is not wired up
on all targets, see

  linux commit 8f6ccf6159aed1f04c6d179f61f6fb2691261e84
  Merge tag 'clone3-v5.3' of ... brauner/linux

  linux commit 8f3220a806545442f6f26195bc491520f5276e7c
  arch: wire-up clone3() syscall

  linux commit 7f192e3cd316ba58c88dfa26796cf77789dd9872
  fork: add clone3
2019-12-30 18:13:56 -05:00
Szabolcs NagyandRich Felker a45289112c add pidfd_open syscall number from linux v5.3
see

  linux commit 7615d9e1780e26e0178c93c55b73309a5dc093d7
  arch: wire-up pidfd_open()

  linux commit 32fcb426ec001cb6d5a4a195091a8486ea77e2df
  pid: add pidfd_open()
2019-12-30 18:12:53 -05:00
Szabolcs NagyandRich Felker f291c09ec9 sys/ptrace.h: add PTRACE_GET_SYSCALL_INFO from linux v5.3
ptrace API to get details of the syscall the tracee is blocked in, see

  linux commit 201766a20e30f982ccfe36bebfad9602c3ff574a
  ptrace: add PTRACE_GET_SYSCALL_INFO request

the align attribute was used to keep the layout the same across targets
e.g. on m68k uint32_t is 2 byte aligned, this helps with compat ptrace.
2019-12-30 18:12:49 -05:00
Szabolcs NagyandRich Felker 06636c5542 sys/socket.h: add SO_DETACH_REUSEPORT_BPF from linux v5.3
see

  linux commit 99f3a064bc2e4bd5fe50218646c5be342f2ad18c
  bpf: net: Add SO_DETACH_REUSEPORT_BPF
2019-12-30 18:12:46 -05:00
Szabolcs NagyandRich Felker 167cbf1316 netinet/if_ether.h: add ETH_P_LLDP from linux v5.3
see

  linux commit c54c2c72b2b90a3ba61b8cad032a578ce2bf5b35
  net: Add a define for LLDP ethertype
2019-12-30 18:12:42 -05:00
Szabolcs NagyandRich Felker af2576e61d netinet/tcp.h: add TCP_TX_DELAY from linux v5.3
see

  linux commit a842fe1425cb20f457abd3f8ef98b468f83ca98b
  tcp: add optional per socket transmit delay
2019-12-30 18:12:40 -05:00
Szabolcs NagyandRich Felker e858063070 math: fix signed int left shift ub in sqrt
Both sqrt and sqrtf shifted the signed exponent as signed int to adjust
the bit representation of the result. There are signed right shifts too
in the code but those are implementation defined and are expected to
compile to arithmetic shift on supported compilers and targets.
2019-10-13 17:46:58 -04:00
Szabolcs NagyandRich Felker db3cc9a319 add Arm to the copyright file
Several math functions are now from the ARM optimized-routines repo
licensed under standard MIT terms and copyrighted by Arm Limited,
so mention this in the COPYRIGHT too.
2019-10-06 20:27:21 -04:00
Szabolcs NagyandRich Felker 33bc7f05fc remove remaining traces of __tls_get_new
Some declarations of __tls_get_new were left in the code, even
though the definition got removed in

  commit 9d44b6460a
  install dynamic tls synchronously at dlopen, streamline access

this can make the build fail with

  ld: lib/libc.so: hidden symbol `__tls_get_new' isn't defined

when libc.so is linked without --gc-sections, because a .hidden
declaration in asm code creates a reference even if the symbol
is not actually used.
2019-09-29 17:59:30 -04:00
Szabolcs NagyandRich Felker ca57795113 math: optimize lrint on 32bit targets
lrint in (LONG_MAX, 1/DBL_EPSILON) and in (-1/DBL_EPSILON, LONG_MIN)
is not trivial: rounding to int may be inexact, but the conversion to
int may overflow and then the inexact flag must not be raised. (the
overflow threshold is rounding mode dependent).

this matters on 32bit targets (without single instruction lrint or
rint), so the common case (when there is no overflow) is optimized by
inlining the lrint logic, otherwise the old code is kept as a fallback.

on my laptop an i486 lrint call is asm:10ns, old c:30ns, new c:21ns
on a smaller arm core: old c:71ns, new c:34ns
on a bigger arm core: old c:27ns, new c:19ns
2019-09-27 11:18:59 -04:00
Szabolcs NagyandRich Felker b678f7b46a arm: fix setjmp and longjmp asm for armv8-a
armv8 removed the coprocessor instructions other than cp14, so
on an armv8 system the related hwcaps should never be set.

new llvm complains about the use of coprocessor instructions in
armv8-a mode (even though they are never executed at runtime),
so ifdef them out when musl is built for armv8.
2019-09-26 00:34:15 -04:00
Szabolcs NagyandRich Felker 9920bf45cc aarch64: add HWCAP2 flags from linux v5.2
AT_HWCAP2 flags, see

  linux commit 671db581815faf17cbedd7fcbc48823a247d90b1
  arm64: Expose DC CVADP to userspace

  linux commit 06a916feca2b262ab0c1a2aeb68882f4b1108a07
  arm64: Expose SVE2 features for userspace
2019-09-11 10:40:41 -04:00
Szabolcs NagyandRich Felker b60763092e add new syscall numbers from linux v5.2
new mount api syscalls were added, same numers on all targets, see

  linux commit a07b20004793d8926f78d63eb5980559f7813404
  vfs: syscall: Add open_tree(2) to reference or clone a mount

  linux commit 2db154b3ea8e14b04fee23e3fdfd5e9d17fbc6ae
  vfs: syscall: Add move_mount(2) to move mounts around

  linux commit 24dcb3d90a1f67fe08c68a004af37df059d74005
  vfs: syscall: Add fsopen() to prepare for superblock creation

  linux commit ecdab150fddb42fe6a739335257949220033b782
  vfs: syscall: Add fsconfig() for configuring and managing a context

  linux commit 93766fbd2696c2c4453dd8e1070977e9cd4e6b6d
  vfs: syscall: Add fsmount() to create a mount for a superblock

  linux commit cf3cba4a429be43e5527a3f78859b1bfd9ebc5fb
  vfs: syscall: Add fspick() to select a superblock for reconfiguration

  linux commit 9c8ad7a2ff0bfe58f019ec0abc1fb965114dde7d
  uapi, x86: Fix the syscall numbering of the mount API syscalls [ver #2]

  linux commit d8076bdb56af5e5918376cd1573a6b0007fc1a89
  uapi: Wire up the mount API syscalls on non-x86 arches [ver #2]
2019-09-11 10:40:39 -04:00
Szabolcs NagyandRich Felker 81e795e1f1 fcntl.h: add AT_RECURSIVE from linux v5.2
apply open_tree with OPEN_TREE_CLONE call to the entire subtree, see

  linux commit a07b20004793d8926f78d63eb5980559f7813404
  vfs: syscall: Add open_tree(2) to reference or clone a mount
2019-09-11 10:40:31 -04:00
Szabolcs NagyandRich Felker cc1a53515b fcntl.h: add AT_STATX_ statx sync flag definitions
see

  linux commit a528d35e8bfcc521d7cb70aaf03e1bd296c8493f
  statx: Add a system call to make enhanced file info available

these are linux specific and not reserved names for fcntl.h so they
are under _BSD_SOURCE|_GNU_SOURCE.
2019-09-11 10:40:11 -04:00
Szabolcs NagyandRich Felker 1e22eb2e8f sched.h: add CLONE_PIDFD from linux v5.2
when set a pidfd is stored in parent_tidptr, see

  linux commit b3e5838252665ee4cfa76b82bdf1198dca81e5be
  clone: add CLONE_PIDFD
2019-09-11 10:40:04 -04:00
Szabolcs NagyandRich Felker b5134ccd4d netinet/if_ether.h: add ETH_P_DSA_8021Q from linux v5.2
ethertype for fake VLAN header for DSA, see

  linux commit bf5bc3ce8a8f32a0d45b6820ede8f9fc3e9c23df
  ether: Add dedicated Ethertype for pseudo-802.1Q DSA tagging
2019-09-11 10:39:58 -04:00
Szabolcs NagyandRich Felker f2435263d7 make relocation time symbol lookup and dlsym consistent
Using common code path for all symbol lookups fixes three dlsym issues:

- st_shndx of STT_TLS symbols were not checked and thus an undefined
  tls symbol reference could be incorrectly treated as a definition
  (the sysv hash lookup returns undefined symbols, gnu does not, so should
  be rare in practice).

- symbol binding was not checked so a hidden symbol may be returned
  (in principle STB_LOCAL symbols may appear in the dynamic symbol table
  for hidden symbols, but linkers most likely don't produce it).

- mips specific behaviour was not applied (ARCH_SYM_REJECT_UND) so
  undefined symbols may be returned on mips.

always_inline is used to avoid relocation performance regression, the
code generation for find_sym should not be affected.
2019-08-12 18:25:38 -04:00
Szabolcs NagyandRich Felker f3f96f2daa add new syscall numbers from linux v5.1
syscall numbers are now synced up across targets (starting from 403 the
numbers are the same on all targets other than an arch specific offset)

IPC syscalls sem*, shm*, msg* got added where they were missing (except
for semop: only semtimedop got added), the new semctl, shmctl, msgctl
imply IPC_64, see

  linux commit 0d6040d4681735dfc47565de288525de405a5c99
  arch: add split IPC system calls where needed

new 64bit time_t syscall variants got added on 32bit targets, see

  linux commit 48166e6ea47d23984f0b481ca199250e1ce0730a
  y2038: add 64-bit time_t syscalls to all 32-bit architectures

new async io syscalls got added, see

  linux commit 2b188cc1bb857a9d4701ae59aa7768b5124e262e
  Add io_uring IO interface

  linux commit edafccee56ff31678a091ddb7219aba9b28bc3cb
  io_uring: add support for pre-mapped user IO buffers

a new syscall got added that uses the fd of /proc/<pid> as a stable
handle for processes: allows sending signals without pid reuse issues,
intended to eventually replace rt_sigqueueinfo, kill, tgkill and
rt_tgsigqueueinfo, see

  linux commit 3eb39f47934f9d5a3027fe00d906a45fe3a15fad
  signal: add pidfd_send_signal() syscall

on some targets (arm, m68k, s390x, sh) some previously missing syscall
numbers got added as well.
2019-07-01 14:20:58 -04:00
Szabolcs NagyandRich Felker 4acfc287d8 ipc: prefer SYS_ipc when it is defined
Linux v5.1 introduced ipc syscalls on targets where previously only
SYS_ipc was available, change the logic such that the ipc code keeps
using SYS_ipc which works backward compatibly on older kernels.

This changes behaviour on microblaze which had both mechanisms, now
SYS_ipc will be used instead of separate syscalls.
2019-07-01 14:17:59 -04:00
Szabolcs NagyandRich Felker b574de5bf3 mips64: fix syscall numbers of io_pgetevents and rseq
the numbers added in

  commit d149e69c02
  add io_pgetevents and rseq syscall numbers from linux v4.18

were incorrect.
2019-07-01 14:17:02 -04:00
Szabolcs NagyandRich Felker 47bc7f0afe elf.h: add NT_ARM_PAC{A,G}_KEYS from linux v5.1
to request or change pointer auth keys for criu via ptrace, new in

  linux commit d0a060be573bfbf8753a15dca35497db5e968bb0
  arm64: add ptrace regsets for ptrauth key management
2019-07-01 14:16:49 -04:00
Szabolcs NagyandRich Felker eea841e653 netinet/in.h: add INADDR_ALLSNOOPERS_GROUP from linux v5.1
RFC 4286: "The IPv4 multicast address for All-Snoopers is 224.0.0.106."
from

  linux commit 4effd28c1245303dce7fd290c501ac2c11052114
  bridge: join all-snoopers multicast address
2019-07-01 14:16:41 -04:00
Szabolcs NagyandRich Felker 5b2b682ddc sys/socket.h: add SO_BINDTOIFINDEX from linux v5.1
SO_BINDTOIFINDEX behaves similar to SO_BINDTODEVICE, but takes a
network interface index as argument, rather than the network
interface name. see

  linux commit f5dd3d0c9638a9d9a02b5964c4ad636f06cf7e2c
  net: introduce SO_BINDTOIFINDEX sockopt
2019-07-01 14:12:36 -04:00
Szabolcs NagyandRich Felker 4523505114 s390x: drop SO_ definitions from bits/socket.h
the s390x definitions matched the generic ones in sys/socket.h.
2019-07-01 14:12:25 -04:00
Szabolcs NagyandRich Felker 93b5f6c843 netinet/in.h: add IPV6_ROUTER_ALERT_ISOLATE from linux v5.1
restricts router alert packets received by the socket to the
socket's namespace only. see

  linux commit 9036b2fe092a107856edd1a3bad48b83f2b45000
  net: ipv6: add socket option IPV6_ROUTER_ALERT_ISOLATE
2019-07-01 14:12:14 -04:00
Szabolcs NagyandRich Felker c7081edcf8 sys/prctl.h: add PR_SPEC_DISABLE_NOEXEC from linux v5.1
allows specifying that the speculative store bypass disable bit should
be cleared on exec. see

  linux commit 71368af9027f18fe5d1c6f372cfdff7e4bde8b48
  x86/speculation: Add PR_SPEC_DISABLE_NOEXEC
2019-07-01 14:12:06 -04:00
Szabolcs NagyandRich Felker 9e8dd1c7f1 fcntl.h: add F_SEAL_FUTURE_WRITE from linux v5.1
needed for android so it can migrate from its ashmem to memfd.
allows making the memfd readonly for future users while keeping
a writable mmap of it. see

  linux commit ab3948f58ff841e51feb845720624665ef5b7ef3
  mm/memfd: add an F_SEAL_FUTURE_WRITE seal to memfd
2019-07-01 14:11:56 -04:00
Szabolcs NagyandRich Felker f67b3c1799 sys/fanotify.h: update for linux v5.1
includes changes from linux v5.1

  linux commit 235328d1fa4251c6dcb32351219bb553a58838d2
  fanotify: add support for create/attrib/move/delete events

  linux commit 5e469c830fdb5a1ebaa69b375b87f583326fd296
  fanotify: copy event fid info to user

  linux commit e9e0c8903009477b630e37a8b6364b26a00720da
  fanotify: encode file identifier for FAN_REPORT_FID

as well as earlier changes that were missed.

sys/statfs.h is included for fsid_t.
2019-07-01 14:11:42 -04:00
Szabolcs NagyandRich Felker a60b9e0686 fix tls offsets when p_vaddr%p_align != 0 on TLS_ABOVE_TP targets
currently the bfd linker does not seem to create tls segments where
p_vaddr%p_align != 0, but this is valid in ELF and then the runtime
computed tls offset must satisfy

  offset%p_align == (base+p_vaddr)%p_align

and in case of local exec tls (main executable) the smallest such
offset must be used (otherwise it is incompatible with the offset
computed by the static linker). the !TLS_ABOVE_TP case is handled
correctly (the offset is negative then in the formula).

the ldso code for TLS_ABOVE_TP is changed so the static tls offset
of each module satisfies the formula.
2019-05-16 21:48:39 -04:00
Szabolcs NagyandRich Felker 6104dae908 fix static tls offsets of shared libs on TLS_ABOVE_TP targets
tls_offset should always point to the end of the allocated static tls
area, but this was not handled correctly on "tls variant 1" targets
in the dynamic linker:

after application tls was allocated, tls_offset was aligned up,
potentially wasting tls space. (alignment may be needed at the
begining of the tls area, not at the end, but that will be fixed
separately as it is unlikely to affect real binaries.)

when static tls was allocated for a shared library, tls_offset was
only updated with the size of the tls segment which does not include
alignment gaps, which can easily happen if the tls size update for
one library leaves tls_offset misaligned for the next one. this can
cause oob access in __copy_tls or arbitrary breakage at tls access.
(the issue was observed on aarch64 with rust binaries)
2019-05-16 20:12:56 -04:00
Szabolcs NagyandRich Felker e4dd65305a math: new pow
from https://github.com/ARM-software/optimized-routines,
commit 04884bd04eac4b251da4026900010ea7d8850edc

The underflow exception is signaled if the result is in the subnormal
range even if the result is exact.

code size change: +3421 bytes.
benchmark on x86_64 before, after, speedup:

-Os:
   pow rthruput: 102.96 ns/call 33.38 ns/call 3.08x
    pow latency: 144.37 ns/call 54.75 ns/call 2.64x
-O3:
   pow rthruput:  98.91 ns/call 32.79 ns/call 3.02x
    pow latency: 138.74 ns/call 53.78 ns/call 2.58x
2019-04-17 23:45:40 -04:00
Szabolcs NagyandRich Felker e16f7b3c02 math: new exp and exp2
from https://github.com/ARM-software/optimized-routines,
commit 04884bd04eac4b251da4026900010ea7d8850edc

TOINT_INTRINSICS and EXP_USE_TOINT_NARROW cases are unused.

The underflow exception is signaled if the result is in the subnormal
range even if the result is exact (e.g. exp2(-1023.0)).

code size change: -1672 bytes.
benchmark on x86_64 before, after, speedup:

-Os:
   exp rthruput:  12.73 ns/call  6.68 ns/call 1.91x
    exp latency:  45.78 ns/call 21.79 ns/call 2.1x
  exp2 rthruput:   6.35 ns/call  5.26 ns/call 1.21x
   exp2 latency:  26.00 ns/call 16.58 ns/call 1.57x
-O3:
   exp rthruput:  12.75 ns/call  6.73 ns/call 1.89x
    exp latency:  45.91 ns/call 21.80 ns/call 2.11x
  exp2 rthruput:   6.47 ns/call  5.40 ns/call 1.2x
   exp2 latency:  26.03 ns/call 16.54 ns/call 1.57x
2019-04-17 23:45:25 -04:00
Szabolcs NagyandRich Felker 2a3210cf4a math: new log2
from https://github.com/ARM-software/optimized-routines,
commit 04884bd04eac4b251da4026900010ea7d8850edc

code size change: +2458 bytes (+1524 bytes with fma).
benchmark on x86_64 before, after, speedup:

-Os:
  log2 rthruput:  16.08 ns/call 10.49 ns/call 1.53x
   log2 latency:  44.54 ns/call 25.55 ns/call 1.74x
-O3:
  log2 rthruput:  15.92 ns/call 10.11 ns/call 1.58x
   log2 latency:  44.66 ns/call 26.16 ns/call 1.71x
2019-04-17 23:45:13 -04:00
Szabolcs NagyandRich Felker 236cd056e8 math: new log
from https://github.com/ARM-software/optimized-routines,
commit 04884bd04eac4b251da4026900010ea7d8850edc

Assume __FP_FAST_FMA implies __builtin_fma is inlined as a single
instruction.

code size change: +4588 bytes (+2540 bytes with fma).
benchmark on x86_64 before, after, speedup:

-Os:
   log rthruput:  12.61 ns/call  7.95 ns/call 1.59x
    log latency:  41.64 ns/call 23.38 ns/call 1.78x
-O3:
   log rthruput:  12.51 ns/call  7.75 ns/call 1.61x
    log latency:  41.82 ns/call 23.55 ns/call 1.78x
2019-04-17 23:44:59 -04:00
Szabolcs NagyandRich Felker d28cd0ad42 math: new powf
from https://github.com/ARM-software/optimized-routines,
commit 04884bd04eac4b251da4026900010ea7d8850edc

POWF_SCALE != 1.0 case only matters if TOINT_INTRINSICS is set, which
is currently not supported for any target.

SNaN is not supported, it would require an issignalingf
implementation.

code size change: -816 bytes.
benchmark on x86_64 before, after, speedup:

-Os:
  powf rthruput:  95.14 ns/call 20.04 ns/call 4.75x
   powf latency: 137.00 ns/call 34.98 ns/call 3.92x
-O3:
  powf rthruput:  92.48 ns/call 13.67 ns/call 6.77x
   powf latency: 131.11 ns/call 35.15 ns/call 3.73x
2019-04-17 23:44:01 -04:00
Szabolcs NagyandRich Felker 3f94c648ef math: new exp2f and expf
from https://github.com/ARM-software/optimized-routines,
commit 04884bd04eac4b251da4026900010ea7d8850edc

In expf TOINT_INTRINSICS is kept, but is unused, it would require support
for __builtin_round and __builtin_lround as single instruction.

code size change: +94 bytes.
benchmark on x86_64 before, after, speedup:

-Os:
  expf rthruput:   9.19 ns/call  8.11 ns/call 1.13x
   expf latency:  34.19 ns/call 18.77 ns/call 1.82x
 exp2f rthruput:   5.59 ns/call  6.52 ns/call 0.86x
  exp2f latency:  17.93 ns/call 16.70 ns/call 1.07x
-O3:
  expf rthruput:   9.12 ns/call  4.92 ns/call 1.85x
   expf latency:  34.44 ns/call 18.99 ns/call 1.81x
 exp2f rthruput:   5.58 ns/call  4.49 ns/call 1.24x
  exp2f latency:  17.95 ns/call 16.94 ns/call 1.06x
2019-04-17 23:43:53 -04:00
Szabolcs NagyandRich Felker 098868b338 math: new log2f
from https://github.com/ARM-software/optimized-routines,
commit 04884bd04eac4b251da4026900010ea7d8850edc

code size change: +177 bytes.
benchmark on x86_64 before, after, speedup:

-Os:
 log2f rthruput:  11.38 ns/call  5.99 ns/call 1.9x
  log2f latency:  35.01 ns/call 22.57 ns/call 1.55x
-O3:
 log2f rthruput:  10.82 ns/call  5.58 ns/call 1.94x
  log2f latency:  35.13 ns/call 21.04 ns/call 1.67x
2019-04-17 23:43:43 -04:00
Szabolcs NagyandRich Felker db505b794c math: new logf
from https://github.com/ARM-software/optimized-routines,
commit 04884bd04eac4b251da4026900010ea7d8850edc,
with minor changes to better fit into musl.

code size change: +289 bytes.
benchmark on x86_64 before, after, speedup:

-Os:
  logf rthruput:   8.40 ns/call  6.14 ns/call 1.37x
   logf latency:  31.79 ns/call 24.33 ns/call 1.31x
-O3:
  logf rthruput:   8.43 ns/call  5.58 ns/call 1.51x
   logf latency:  32.04 ns/call 20.88 ns/call 1.53x
2019-04-17 23:42:38 -04:00
Szabolcs NagyandRich Felker 169fc008d8 math: add configuration macros
Musl currently aims to support non-nearest rounding mode and does not
support SNaNs. These macros allow marking relevant code paths in case
these decisions are changed later (they also help documenting the
corner cases involved).
2019-04-17 13:12:21 -04:00
Szabolcs NagyandRich Felker 3c3763fc52 math: add macros for static branch prediction hints
These don't have an effectw with -Os so not useful with default settings
other than documenting the expectation.

With --enable-optimize=internal,malloc,string,math the libc.so code size
increases by 18K on x86_64 and performance varies in -2% .. +10%.
2019-04-17 13:12:08 -04:00
Szabolcs NagyandRich Felker 4f8acf953c math: add double precision error handling functions 2019-04-17 13:08:45 -04:00
Szabolcs NagyandRich Felker 9ef6ca4235 math: add single precision error handling functions
These are supposed to be used in tail call positions when handling
special cases in new code. (fp exceptions may be raised "naturally"
by the common code path if special casing is more effort.)

This implements the error handling apis used in
https://github.com/ARM-software/optimized-routines
without errno setting.
2019-04-17 13:08:42 -04:00
Szabolcs NagyandRich Felker fe54544f05 math: add eval_as_float and eval_as_double
Previously type casts or assignments were used for handling excess
precision, which assumed standard C99 semantics, but since it's a
rarely needed obscure detail, it's better to use explicit helper
functions to document where we rely on this.  It also helps if the
code is used outside of the libc in non-C99 compilation mode: with the
default excess precision handling of gcc, explicit inline asm barriers
are needed for narrowing on FLT_EVAL_METHOD!=0 targets.

I plan to use this in new code with the existing style that uses
double_t and float_t as much as possible.

One ugliness is that it is required for almost every return statement
since that does not drop excess precision (the standard changed this
in C11 annex F, but that does not help in non-standard compilation
modes or with old compilers).
2019-04-17 13:07:29 -04:00
Szabolcs NagyandRich Felker b50d315fd2 math: add fp_arch.h with fp_barrier and fp_force_eval
C99 has ways to support fenv access, but compilers don't implement it
and assume nearest rounding mode and no fp status flag access. (gcc has
-frounding-math and then it does not assume nearest rounding mode, but
it still assumes the compiled code itself does not change the mode.
Even if the C99 mechanism was implemented it is not ideal: it requires
all code in the library to be compiled with FENV_ACCESS "on" to make it
usable in non-nearest rounding mode, but that limits optimizations more
than necessary.)

The math functions should give reasonable results in all rounding modes
(but the quality may be degraded in non-nearest rounding modes) and the
fp status flag settings should follow the spec, so fenv side-effects are
important and code transformations that break them should be prevented.

Unfortunately compilers don't give any help with this, the best we can
do is to add fp barriers to the code using volatile local variables
(they create a stack frame and undesirable memory accesses to it) or
inline asm (gcc specific, requires target specific fp reg constraints,
often creates unnecessary reg moves and multiple barriers are needed to
express that an operation has side-effects) or extern call (only useful
in tail-call position to avoid stack-frame creation and does not work
with lto).

We assume that in a math function if an operation depends on the input
and the output depends on it, then the operation will be evaluated at
runtime when the function is called, producing all the expected fenv
side-effects (this is not true in case of lto and in case the operation
is evaluated with excess precision that is not rounded away). So fp
barriers are needed (1) to prevent the move of an operation within a
function (in case it may be moved from an unevaluated code path into an
evaluated one or if it may be moved across a fenv access), (2) force the
evaluation of an operation for its side-effect when it has no input
dependency (may be constant folded) or (3) when its output is unused. I
belive that fp_barrier and fp_force_eval can take care of these and they
should not be needed in hot code paths.
2019-04-17 13:06:43 -04:00
Szabolcs NagyandRich Felker f107d34e76 math: remove sun copyright from libm.h
Nothing is left from the original fdlibm header nor from the bsd
modifications to it other than some internal api declarations.

Comments are dropped that may be copyrightable content.
2019-04-17 13:04:49 -04:00
Szabolcs NagyandRich Felker d59e504279 math: add asuint, asuint64, asfloat and asdouble
Code generation for SET_HIGH_WORD slightly changes, but it only affects
pow, otherwise the generated code is unchanged.
2019-04-17 13:04:28 -04:00
Szabolcs NagyandRich Felker 2d72b58070 math: move complex math out of libm.h
This makes it easier to build musl math code with a compiler that
does not support complex types (tcc) and in general more sensible
factorization of the internal headers.
2019-04-17 13:03:52 -04:00
Szabolcs NagyandRich Felker e980ca7a57 define FP_FAST_FMA* when fma* can be inlined
FP_FAST_FMA can be defined if "the fma function generally executes about
as fast as, or faster than, a multiply and an add of double operands",
which can only be true if the fma call is inlined as an instruction.

gcc sets __FP_FAST_FMA if __builtin_fma is inlined as an instruction,
but that does not mean an fma call will be inlined (e.g. it is defined
with -fno-builtin-fma), other compilers (clang) don't even have such
macro, but this is the closest we can get.

(even if the libc fma implementation is a single instruction, the extern
call overhead is already too big when the macro is used to decide between
x*y+z and fma(x,y,z) so it cannot be based on libc only, defining the
macro unconditionally on targets which have fma in the base isa is also
incorrect: the compiler might not inline fma anyway.)

this solution works with gcc unless fma inlining is explicitly turned off.
2019-04-17 13:02:47 -04:00
Szabolcs NagyandRich Felker 2a03b0b518 aarch64: add HWCAP_ definitions from linux v5.0
HWCAP_SB - speculation barrier instruction available added in linux
commit bd4fb6d270bc423a9a4098108784f7f9254c4e6d
HWCAP_PACA, HWCAP_PACG - pointer authentication instructions available
(address and generic) added in linux commit
7503197562567b57ec14feb3a9d5400ebc56812f
2019-03-13 12:35:02 -04:00
Szabolcs NagyandRich Felker cff993cf32 sys/prctl.h: add PR_PAC_RESET_KEYS from linux v5.0
aarch64 pointer authentication code related prctl that allows
reinitializing the key for the thread, added in linux commit
ba830885656414101b2f8ca88786524d4bb5e8c1
2019-03-13 12:34:56 -04:00
Szabolcs NagyandRich Felker 843e453c11 elf.h: add NT_ definitions from linux v5.0
NT_MIPS_MSA for ptrace access to mips simd arch reg set, added in linux
commit 3cd640832894b85b5929d5bda74505452c800421
NT_ARM_PAC_MASK for ptrace access to pointer auth code mask, added in
commit ec6e822d1a22d0eef1d1fa260dff751dba9a4258
2019-03-13 12:34:08 -04:00
Szabolcs NagyandRich Felker 02f769b20d elf.h: update with C-SKY definitions
C-SKY support was added to binutils 2.32 in commit
b8891f8d622a31306062065813fc278d8a94fe21
the elf.h change was added to glibc 2.29 in commit
4975f0c3d0131fdf697be0b1631c265e5fd39088
2019-03-13 12:34:03 -04:00
Szabolcs NagyandRich Felker 1114df5d60 aarch64, or1k: add kexec_file_load syscall number from linux v5.0
added in linux commit 4e21565b7fd4d9045765f697887e74a704135fe2
2019-03-13 12:33:58 -04:00
Szabolcs NagyandRich Felker 9e06514e23 netinet/tcp.h: add TCP_NLA_SRTT from linux v5.0
smoothed RTT for SCM_TIMESTAMPING_OPT_STATS control messages.
added in linux commit e8bd8fca6773ef49390269bd467bf940a0841ccf
2019-03-13 12:33:54 -04:00
Szabolcs NagyandRich Felker 292bda7ceb netinet/udp.h: add UDP_GRO from linux v5.0
sockopt to enable gro for udp.
added in linux commit e20cf8d3f1f763ad28a9cb3b41305b8a8a42653e
2019-03-13 12:33:51 -04:00
Szabolcs NagyandRich Felker a86c4724c0 powerpc: add PTRACE_SYSEMU from linux v4.20
added in linux commit 5521eb4bca2db733952f068c37bdf3cd656ad23c
2019-03-13 12:33:41 -04:00
Szabolcs NagyandRich Felker 65735baa75 aarch64: add HWCAP_SSBS from linux v4.20
for armv8.5 speculative store bypass PSTATE bit support,
added in linux commit d71be2b6c0e19180b5f80a6d42039cc074a693a2
2019-03-13 12:33:36 -04:00
Szabolcs NagyandRich Felker 59904de7e0 bits/ioctl.h: add TIOC{G,S}ISO7816 from linux v4.20
ISO7816 smart cards ioctls.
linux commit ad8c0eaa0a418ae8ef3f9217638bb86439399eac

the actual kernel definitions are

 #define TIOCGISO7816 _IOR('T', 0x42, struct serial_iso7816)
 #define TIOCSISO7816 _IOWR('T', 0x43, struct serial_iso7816)

where struct serial_iso7816 is defined in linux/serial.h as

struct serial_iso7816 {
	__u32   flags;
	__u32   tg;
	__u32   sc_fi;
	__u32   sc_di;
	__u32   clk;
	__u32   reserved[5];
};
2019-03-13 12:33:15 -04:00
Szabolcs NagyandRich Felker 13c73da72f sys/prctl.h: add PR_SPEC_INDIRECT_BRANCH from linux v4.20
prctls to allow per task control of indirect branch speculation on x86.

added in linux commit 9137bb27e60e554dab694eafa4cca241fa3a694f
2019-03-13 12:33:04 -04:00
Szabolcs NagyandRich Felker 0b026bbd69 netinet/in.h add IPV6_MULTICAST_ALL from linux v4.20
ipv6 analogue of IP_MULTICAST_ALL sockopt.

added in linux commit 15033f0457dca569b284bef0c8d3ad55fb37eacb
2019-03-13 12:32:59 -04:00
Szabolcs NagyandRich Felker c6a61df183 add PACKET_IGNORE_OUTGOING sockopt from linux v4.20
new in linux commit fa788d986a3aac5069378ed04697bd06f83d3488
2019-03-13 12:32:19 -04:00
Szabolcs NagyandRich Felker db8b53f207 sys/mman.h: add new hugetlb mmap flags from linux v4.19
aarch64 supports 32MB and 512MB hugetlb page sizes too.
added in linux commit 20916d4636a9b3c1bf562b305f91d126771edaf9
2019-03-13 12:32:14 -04:00
Szabolcs NagyandRich Felker d66ae1aeaf arm: add io_pgetevents syscall number from v4.19
wired up in linux commit 73aeb2cbcdc9be391b3d32a55319a59ce425426f
2019-03-13 12:32:08 -04:00
Szabolcs NagyandRich Felker 53bef5c55e aarch64, or1k: define rseq syscall number following linux v4.19
added in linux commit db7a2d1809a5b6b08d138ff68837f805fc073351
2019-03-13 12:31:22 -04:00
Szabolcs NagyandRich Felker 5640bc5bd5 elf.h: add new mips core dump note values from linux v4.19
NT_MIPS_FP_MODE is new in linux commit
1ae22a0e35636efceab83728ba30b013df761592

NT_MIPS_DSP is new in linux commit
44109c60176ae73924a42a6bef64ef151aba9095
2019-03-13 12:31:13 -04:00
Szabolcs NagyandRich Felker 3e52f99308 netinet/udp.h: add UDP_ENCAP_RXRPC from linux v4.19
used for optimizing the rxrpc protocol
added in linux commit 5271953cad31b97dea80f848c16e96ad66401199
2019-03-13 12:30:21 -04:00
Szabolcs NagyandRich Felker b47723ab3b netinet/tcp.h: add tcp_info fields from linux v4.19
new fields for RFC 4898 tcp stats in linux
tcpi_bytes_sent added in commit ba113c3aa79a7f941ac162d05a3620bdc985c58d
tcpi_bytes_retrans added in commit fb31c9b9f6c85b1bad569ecedbde78d9e37cd87b
tcpi_dsack_dups added in commit 7e10b6554ff2ce7f86d5d3eec3af5db8db482caa
tcpi_reord_seen added in commit 7ec65372ca534217b53fd208500cf7aac223a383

The new fields change the size of a public struct and thus an ABI break,
but this is how the getsockopt TCP_INFO api is designed: the tcp_info
type must only be used with a length parameter in extern interfaces.
2019-03-13 12:30:12 -04:00
Szabolcs NagyandRich Felker df4b017bcd sys/inotify.h: add IN_MASK_CREATE from linux v4.19
inotify_add_watch flag to prevent modifying existing watch descriptors,
when used on an already watched inode it fails with EEXIST.
added in linux commit 4d97f7d53da7dc830dbf416a3d2a6778d267ae68
2019-03-13 12:29:27 -04:00
Szabolcs NagyandRich Felker 31b3efbf3e sys/socket.h: add SO_TXTIME from linux v4.19
added in linux commit 80b14dee2bea128928537d61c333f24cb8cbb62f
2019-03-13 12:29:23 -04:00
Szabolcs NagyandRich Felker 1de4788610 add NT_VMCOREDD to elf.h from linux v4.18
used for device driver dump in /proc/vmcore
new in linux commit 2724273e8fd00b512596a77ee063f49b25f36507
2018-12-09 22:30:05 -05:00
Szabolcs NagyandRich Felker 836022b1c3 add AT_MINSIGSTKSZ to elf.h from linux v4.18
new in linux commit 94b07c1f8c39c6d839df35fa28ffd1785d385897

currently only supported on aarch64
2018-12-09 22:29:44 -05:00
Szabolcs NagyandRich Felker d149e69c02 add io_pgetevents and rseq syscall numbers from linux v4.18
io_pgetevents is new in linux commit
7a074e96dee62586c935c80cecd931431bfdd0be

rseq is new in linux commit
d7822b1e24f2df5df98c76f0e94a5416349ff759
2018-12-09 22:29:12 -05:00
Szabolcs NagyandRich Felker ca73bea05d add TRAP_UNK si_code to signal.h from linux v4.18
used for undiagnosed trap exceptions where linux previously set si_code
to 0. new in linux commit db78e6a0a6f9f7d7277965600eeb1a5b3a6f55a8
2018-12-09 22:29:07 -05:00
Szabolcs NagyandRich Felker a71276e258 add SIGSYS support to sys/signalfd.h from linux v4.18
new in linux commit 76b7f670730e87974f71df9f6129811e2769666e

in struct signalfd_siginfo the pad member is changed to __pad to keep
the namespace clean, it's not part of the public api.
2018-12-09 22:25:21 -05:00
Szabolcs NagyandRich Felker 0a9a5c20c7 add AF_XDP to sys/socket.h from linux v4.18
new address family and related macros were added in linux commit
68e8b849b221b37a78a110a0307717d45e3593a0
2018-12-09 22:24:12 -05:00
Szabolcs NagyandRich Felker 4b9b8c8ee6 update netinet/udp.h for linux v4.18
add UDP_NO_CHECK6_* to restrict zero UDP6 checksums, new in linux commit
1c19448c9ba6545b80ded18488a64a7f3d8e6998 (pre-v4.18 change, was missed)
add UDP_SEGMENT to support generic segmentation offload for udp datagrams,
bec1f6f697362c5bc635dacd7ac8499d0a10a4e7 (new in v4.18)
2018-12-09 22:23:48 -05:00
Szabolcs NagyandRich Felker ac1af87e1d update netinet/tcp.h for linux v4.18
add packet delivery info to tcp_info,
new in linux commit feb5f2ec646483fb66f9ad7218b1aad2a93a2a5c
add TCP_ZEROCOPY_RECEIVE socket option for zerocopy receive,
new in linux commit 05255b823a6173525587f29c4e8f1ca33fd7677d
add TCP_INQ socket option and TCP_CM_INQ cmsg to get in-queue bytes in cmsg
upon read, new in linux commit b75eba76d3d72e2374fac999926dafef2997edd2
add TCP_REPAIR_* to fix repair socket window probe patch,
new in linux commit 31048d7aedf31bf0f69c54a662944632f29d82f2
2018-12-09 22:23:30 -05:00
Szabolcs NagyandRich Felker e901613888 x86_64: add single instruction fma
fma is only available on recent x86_64 cpus and it is much faster than
a software fma, so this should be done with a runtime check, however
that requires more changes, this patch just adds the code so it can be
tested when musl is compiled with -mfma or -mfma4.
2018-10-15 14:45:28 -04:00
Szabolcs NagyandRich Felker 7396ef0a05 arm: add single instruction fma
vfma is available in the vfpv4 fpu and above, the ACLE standard feature
test for double precision hardware fma support is
  __ARM_FEATURE_FMA && __ARM_FP&8
we need further checks to work around clang bugs (fixed in clang >=7.0)
  && !__SOFTFP__
because __ARM_FP is defined even with -mfloat-abi=soft
  && !BROKEN_VFP_ASM
to disable the single precision code when inline asm handling is broken.

For runtime selection the HWCAP_ARM_VFPv4 hwcap flag can be used, but
that requires further work.
2018-10-15 14:42:46 -04:00
Szabolcs NagyandRich Felker 7c5f3bb955 powerpc: add single instruction fabs, fabsf, fma, fmaf, sqrt, sqrtf
These are only available on hard float target and sqrt is not available
in the base ISA, so further check is used.
2018-10-15 14:41:59 -04:00
Szabolcs NagyandRich Felker 1da534ada8 s390x: add single instruction fma and fmaf
These are available in the s390x baseline isa -march=z900.
2018-10-15 14:41:43 -04:00
Szabolcs NagyandRich Felker 7b384c42b7 fix fesetround error checking
Rounding modes are not bit flags, but arbitrary non-negative integers.
2018-10-10 16:15:51 -04:00
Szabolcs NagyandRich Felker c50985d5c8 new tsearch implementation
Rewrote the AVL tree implementation:

- It is now non-recursive with fixed stack usage (large enough for
worst case tree height). twalk and tdestroy are still recursive as
that's smaller/simpler.

- Moved unrelated interfaces into separate translation units.

- The node structure is changed to use indexed children instead of
left/right pointers, this simplifies the balancing logic.

- Using void * pointers instead of struct node * in various places,
because this better fits the api (node address is passed in a void**
argument, so it is tempting to incorrectly cast it to struct node **).

- As a further performance improvement the rebalancing now stops
when it is not needed (subtree height is unchanged). Otherwise
the behaviour should be the same as before (checked over generated
random inputs that the resulting tree shape is equivalent).

- Removed the old copyright notice (including prng related one: it
should be licensed under the same terms as the rest of the project).

.text size of pic tsearch + tfind + tdelete + twalk:

   x86_64 i386 aarch64  arm mips powerpc ppc64le  sh4 m68k s390x
old   941  899    1220 1068 1852    1400    1600 1008 1008  1488
new   857  881    1040  976 1564    1192    1360  736  820  1408
2018-09-20 17:57:47 -04:00
Szabolcs NagyandRich Felker 5ef60206ff add arm and sh bits/ptrace.h
These should have been added in commit
df6d9450ea
that added target specific PTRACE_ macros, but somehow got missed.
2018-09-20 17:56:29 -04:00
Szabolcs NagyandRich Felker 056f9d818e rewrite __aeabi_read_tp in asm
__aeabi_read_tp used to call c code, but that was incorrect as the
arm runtime abi specifies special pcs for this function: it is only
allowed to clobber r0, ip, lr and cpsr.

maintainer's note: the old code explicitly saved and restored all
general-purpose registers which are call-clobbered in the normal
calling convention, so it's unlikely that any real-world compilers
produced code that could break. however theoretically they could have
chosen to use floating point registers, in which case the caller's
values of those registers would be clobbered.
2018-08-28 17:08:14 -04:00
Szabolcs NagyandRich Felker c06c1eb661 fix tls access on arm targets before armv6k
commit 610c5a8524 changed the thread
pointer setup so tp points at the end of the pthread struct on arm,
but failed to update __aeabi_read_tp so it was off by 8.

this broke tls access in code that is compiled with -mtp=soft, which
is the default when target arch is pre armv6k or thumb1.

maintainer's note: no release versions are affected.
2018-08-23 16:41:44 -04:00
Szabolcs NagyandRich Felker df6d9450ea add support for arch-specific ptrace command macros
sys/ptrace.h is target specific, use bits/ptrace.h to add target
specific macro definitions.

these macros are kept in the generic sys/ptrace.h even though some
targets don't support them:

PTRACE_GETREGS
PTRACE_SETREGS
PTRACE_GETFPREGS
PTRACE_SETFPREGS
PTRACE_GETFPXREGS
PTRACE_SETFPXREGS

so no macro definition got removed in this patch on any target. only
s390x has a numerically conflicting macro definition (PTRACE_SINGLEBLOCK).

the PT_ aliases follow glibc headers, otherwise the definitions come
from linux uapi headers except ones that are skipped in glibc and
there is no real kernel support (s390x PTRACE_*_AREA) or need special
type definitions (mips PTRACE_*_WATCH_*) or only relevant for linux
2.4 compatibility (PTRACE_OLDSETOPTIONS).
2018-07-17 22:25:08 -04:00
Szabolcs NagyandRich Felker 7ea32a8282 sys/ptrace.h: add missing PTRACE_EVENT_STOP
new in linux v3.1 commit 3544d72a0e10d0aa1c1bd59ed77a53a59cdc12f7
changed in linux v3.4 commit 5cdf389aee90109e2e3d88085dea4dd5508a3be7

A tracer recieves this event in the waitpid status of a PTRACED_SEIZED
process.
2018-07-17 22:24:15 -04:00
Szabolcs NagyandRich Felker cbf16c4abf uchar.h: define char16_t and char32_t for old c++
including uchar.h in c++ code is only well defined in c++11 onwards
where char16_t and char32_t type definitions must be hidden since they
are keywords.  however some c++ code compiled for older c++ standard
include uchar.h too and they need the typedefs, this fix makes such
code work.
2018-07-17 22:24:03 -04:00
Szabolcs NagyandRich Felker 38f2fa3d02 add memfd_create syscall wrapper
memfd_create was added in linux v3.17 and glibc has api for it.
2018-06-20 21:36:04 -04:00
Szabolcs NagyandRich Felker b64d66d0b0 add mlock2 linux syscall wrapper
mlock2 syscall was added in linux v4.4 and glibc has api for it.
It falls back to mlock in case of flags==0, so that case works
even on older kernels.

MLOCK_ONFAULT is moved under _GNU_SOURCE following glibc.
2018-06-20 21:35:45 -04:00