1. as reported by William Haddon, the value returned by snprintf was
wrongly used as a length passed to sendto, despite it possibly
exceeding the buffer length. this could lead to invalid reads and
leaking additional data to syslog.
2. openlog was storing a pointer to the ident string passed by the
caller, rather than copying it. this bug is shared with (and even
documented in) other implementations like glibc, but such behavior
does not seem to meet the requirements of the standard.
3. extremely long ident provided to openlog, or corrupt ident due to
the above issue, could possibly have resulted in buffer overflows.
despite having the potential for smashing the stack, i believe the
impact is low since ident points to a short string literal in typical
application usage (and per the above bug, other usages will break
horribly on other implementations).
4. when used with LOG_NDELAY, openlog was not connecting the
newly-opened socket; sendto was being used instead. this defeated the
main purpose of LOG_NDELAY: preparing for chroot.
5. the default facility was not being used at all, so all messages
without an explicit facility passed to syslog were getting logged at
the kernel facility.
6. setlogmask was not thread-safe; no synchronization was performed
updating the mask. the fix uses atomics rather than locking to avoid
introducing a lock in the fast path for messages whose priority is not
in the mask.
7. in some code paths, the syslog lock was being unlocked twice; this
could result in releasing a lock that was actually held by a different
thread.
some additional enhancements to syslog such as a default identifier
based on argv[0] or similar may still be desired; at this time, only
the above-listed bugs have been fixed.
it serves no purpose (binaries linked against musl as -lc/libc.so
automatically get the right DT_NEEDED value of libc.so) and causes
ldconfig to misbehave (making a symlink to ld-musl named libc.so in
/lib). ldconfig is not used on pure musl systems, but if ld-musl is
installed on a system where it's not the primary libc, this will
pollute the system /lib with a symlink to musl named libc.so, which
should NOT exist and could cause problems linking native apps. also,
the existence of the soname caused spurious warnings from ldconfig
when /lib and /usr/lib were the same physical directory.
this fix is far from ideal and breaks the rule of not using
arch-specific #ifdefs, but for now we just need a solution to the
existing breakage.
the underlying problem is that the kernel folks made a very stupid
decision to make misalignment of this struct part of the kernel
API/ABI for x86_64, in order to avoid writing a few extra lines of
code to handle both 32- and 64-bit userspace on 64-bit kernels. I had
just added the packed attribute unconditionally thinking it was
harmless on 32-bit archs, but non-x86 32-bit archs have 8-byte
alignment on 64-bit types.
wctype_t was incorrectly "int" rather than "long" on x86_64. not only
is this an ABI incompatibility; it's also a major design flaw if we
ever wanted wctype_t to be implemented as a pointer, which would be
necessary if locales support custom character classes, since int is
too small to store a converted pointer. this commit fixes wctype_t to
be unsigned long on all archs, matching the LSB ABI; this change does
not matter for C code, but for C++ it affects mangling.
the same issue applied to wctrans_t. glibc/LSB defines this type as
const __int32_t *, but since no such definition is visible, I've just
expanded the definition, int, everywhere.
it would be nice if these types (which don't vary by arch) could be in
wctype.h, but the OB XSI requirement in POSIX that wchar.h expose some
types and functions from wctype.h precludes doing so. glibc works
around this with some hideous hacks, but trying to duplicate that
would go against the intent of musl's headers.
lenl-lenr is not a valid expression for a signed int return value from
strverscmp, since after implicit conversion from size_t to int this
difference could have the wrong sign or might even be zero. using the
difference for char values works since they're bounded well within the
range of differences representable by int, but it does not work for
size_t values.
1. wrong return value and missing errno for negative suffix len
2. failure to catch suffix len > strlen
3. remove unwanted clearing of input string in invalid case
based on patch contributed by Anthony G. Basile (blueness)
some issues remain with the filename generation algorithm and other
small bugs, but this patch has been sitting around long enough that I
feel it's best to get it committed and then work out any remaining
issues.
patch by Jens Gustedt.
previously, the intended policy was to use __environ in code that must
conform to the ISO C namespace requirements, and environ elsewhere.
this policy was not followed in practice anyway, making things
confusing. on top of that, Jens reported that certain combinations of
link-time optimization options were breaking with the inconsistent
references; this seems to be a compiler or linker bug, but having it
go away is a nice side effect of the changes made here.
based on patch by Isaac Dunham, moved to its own file to avoid
increasing bss on static linked programs not using this nonstandard
function but using the standard getgrent function, and vice versa.
this definitely has the potential to be a bikeshed topic, so some
justification is in order. most of the changes made fit into one of
the following categories:
1. alignment with text in posix, xsh 2.3
2. eliminating overly-specific text for shared error codes
3. making the message match more closely with the macro name
4. removing extraneous words
in particular, the EAGAIN/EWOULDBLOCK text is updated to match the
description of EAGAIN (which covers both uses) rather than saying the
operation would block, and ENOTSUP/EOPNOTSUPP is updated not to
mention sockets.
the distinction between ENFILE/EMFILE has also been clarified; ENFILE
is aligned with the posix text, and EMFILE, which lacks concise posix
text matching any historic message, is updated to emphasize that the
exhausted resource is not open files/open file descriptions, but
rather the integer 'address space' of file descriptors.
some messages may be further tweaked based on feedback.
read should never return anything but 0 or sizeof ec here, but if it
does, we want to treat any other return as "success". then the caller
will get back the pid and is responsible for waiting on it when it
immediately exits.
the proposed change was described in detail in detail previously on
the mailing list. in short, vfork is unsafe because:
1. the compiler could make optimizations that cause the child to
clobber the parent's local vars.
2. strace is buggy and allows the vforking parent to run before the
child execs when run under strace.
the new design uses a close-on-exec pipe instead of vfork semantics to
synchronize the parent and child so that the parent does not return
before the child has finished using its arguments (and now, also its
stack). this also allows reporting exec failures to the caller instead
of giving the caller a child that mysteriously exits with status 127
on exec error.
basic testing has been performed on both the success and failure code
paths. further testing should be done.
also, don't waste code/time on F_GETFL since pipes always have blank
flags initially (at least on old kernels, which are all this fallback
code matters for).
this change shaves ~1k off libc.so bss size, and also avoids hard
errors in the case where the static buffer was not large enough to
hold the result.
this whole framework is really ugly and might should be replaced or at
least heavily overhauled when some changes/factorizations are made to
getaddrinfo internals in the future.
this bug seems to have been introduced when the map_library signatures
was changed to return the mapping in a temp dso structure instead of
into separate variables.
the main goal of these changes is to address the case where an
application provides a stack of size N, but TLS has size M that's a
significant portion of the size N (or even larger than N), thus giving
the application less stack space than it expected or no stack at all!
the new strategy pthread_create now uses is to only put TLS on the
application-provided stack if TLS is smaller than 1/8 of the stack
size or 2k, whichever is smaller. this ensures that the application
always has "close enough" to what it requested, and the threshold is
chosen heuristically to make sure "sane" amounts of TLS still end up
in the application-provided stack.
if TLS does not fit the above criteria, pthread_create uses mmap to
obtain space for TLS, but still uses the application-provided stack
for actual call frame stack. this is to avoid wasting memory, and for
the sake of supporting ugly hacks like garbage collection based on
assumptions that the implementation will use the provided stack range.
in order for the above heuristics to ever succeed, the amount of TLS
space wasted on POSIX TSD (pthread_key_create based) needed to be
reduced. otherwise, these changes would preclude any use of
pthread_create without mmap, which would have serious memory usage and
performance costs for applications trying to create huge numbers of
threads using pre-allocated stack space. the new value of
PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should
still be plenty more than real-world applications need, especially now
that C11/gcc-style TLS is now supported in musl, and most apps and
libraries choose to use that instead of POSIX TSD when available.
at the same time, PTHREAD_STACK_MIN has been decreased. it was
originally set to PAGE_SIZE back when there was no support for TLS or
application-provided stacks, and requests smaller than a whole page
did not make sense. now, there are two good reasons to support
requests smaller than a page: (1) applications could provide
pre-allocated stacks smaller than a page, and (2) with smaller stack
sizes, stack+TLS+TSD can all fit in one page, making it possible for
applications which need huge numbers of threads with minimal stack
needs to allocate exactly one page per thread. the new value of
PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for
sigaltstack.
this should generate faster and smaller code, especially with inline
syscalls. the conditional with cnt is ugly, but thankfully cnt is
always a constant anyway so it gets evaluated at compile time. it may
be preferable to make separate __wake and __wakeall macros without a
count argument.
priv flag is not used yet; private futex support still needs to be
done at some point in the future.
it's not clear to me at the moment whether the code that was removed
(and which is now being re-added) is needed, but it's far from being a
no-op, and i don't want to risk breaking regex in this release.
alternatively, we could define it in sys/socket.h since SO* is
reserved there, and tcp.h includes sys/socket.h in extensions mode.
note that SOL_TCP is simply wrong and it's only here for compatibility
with broken applications. the correct argument to pass for setting TCP
socket options is IPPROTO_TCP, which of course has the same value as
SOL_TCP but works everywhere.
this is a trivial no-op, because dlclose never deletes libraries. thus
we might as well have it in the header in case some application wants
it, since we're already providing it anyway.
based on patch by Pierre Carrier <[email protected]> that just added
the flag constant, but with minimal additional code so that it
actually works as documented. this is a nonstandard option but some
major software (reportedly, Firefox) uses it and it was easy to add
anyway.
the historical mess of having different definitions for C and C++
comes from the historical C definition as (void *)0 and the fact that
(void *)0 can't be used in C++ because it does not convert to other
pointer types implicitly. however, using plain 0 in C++ exposed bugs
in C++ programs that call variadic functions with NULL as an argument
and (wrongly; this is UB) expect it to arrive as a null pointer. on
64-bit machines, the high bits end up containing junk. glibc dodges
the issue by using a GCC extension __null to define NULL; this is
observably non-conforming because a conforming application could
observe the definition of NULL via stringizing and see that it is
neither an integer constant expression with value zero nor such an
expression cast to void.
switching to 0L eliminates the issue and provides compatibility with
broken applications, since on all musl targets, long and pointers have
the same size, representation, and argument-passing convention. we
could maintain separate C and C++ definitions of NULL (i.e. just use
0L on C++ and use (void *)0 on C) but after careful analysis, it seems
extremely difficult for a C program to even determine whether NULL has
integer or pointer type, much less depend in subtle, unintentional
ways, on whether it does. C89 seems to have no way to make the
distinction. on C99, the fact that (int)(void *)0 is not an integer
constant expression, along with subtle VLA/sizeof semantics, can be
used to make the distinction, but many compilers are non-conforming
and give the wrong result to this test anyway. on C11, _Generic can
trivially make the distinction, but it seems unlikely that code
targetting C11 would be so backwards in caring which definition of
NULL an implementation uses.
as such, the simplest path of using the same definition for NULL in
both C and C++ was chosen. the #undef directive was also removed so
that the compiler can catch and give a warning or error on
redefinition if buggy programs have defined their own versions of
NULL prior to inclusion of standard headers.
this is wasteful and useless from a standpoint of sane programs, but
it is required by the standard, and the current requirements were
upheld with the closure of Austin Group issue #639:
http://austingroupbugs.net/view.php?id=639
the old definitions were wrong on some archs. actually, EPOLL_NONBLOCK
probably should not even be defined; it is not accepted by the kernel
and it's not clear to me whether it has any use at all, even if it did
work. this issue should be revisited at some point, but I'm leaving it
in place for now in case some applications reference it.
libc is the macro, __libc is the internal symbol, but under some
configurations on old/broken compilers, the symbol might not actually
exist and the libc macro might instead use __libc_loc() to obtain
access to the object.
the previous logic was assuming the kernel would give EINVAL when
passed an invalid address, but instead with MAP_FIXED it was giving
EPERM, as it considered this an attempt to map over kernel memory.
instead of trying to get the kernel to do the rigth thing, the new
code just handles the error in userspace.
I have also cleaned up the code to use a single mask to check for
invalid low bits and unsupported high bits, so it's simpler and more
clearly correct. the old code was actually wrong for sizeof(long)
smaller than sizeof(off_t) but not equal to 4; now it should be
correct for all possibilities.
for 64-bit systems, the low-bits test is new and extraneous (the
kernel should catch the error anyway when the mmap2 syscall is not
used), but it's cheap anyway. if this is an issue, the OFF_MASK
definition could be tweaked to omit the low bits when SYS_mmap2 is not
defined.
despite glibc using __key and __seq rather than key and seq, some
applications, notably busybox, assume the names are key and seq unless
glibc is being used. and the names key and seq are really the ones
that _should_ be exposed when not attempting to present a
standards-conforming namespace; apps should not be using names that
begin with double-underscore. thus, the optimal fix is to use key and
seq as the actual names of the members when in bsd/gnu source profile,
and define macros for __key and __seq that redirect to plain key and
seq.
traditionally, both BSD and GNU systems have it this way.
sys/syscall.h is purely syscall number macros. presently glibc exposes
the syscall declaration in unistd.h only with _GNU_SOURCE, but that
does not reflect historical practice.
a while back, gcc switched from using the old _init/_fini fragments
method for calling ctors and dtors on arm to the __init_array and
__fini_array method. unfortunately, on glibc this depends on ugly
hacks involving making libc.so a linker script and pulling parts of
libc into the main program binary. so I cheat a little bit, and just
write asm to iterate over the init/fini arrays from the _init/_fini
asm. the same approach could be used on any arch it's needed on, but
for now arm is the only one.
this change fixes an obscure issue with some nonstandard kernels,
where the initial brk syscall returns a pointer just past the end of
bss rather than the beginning of a new page. in that case, the dynamic
linker has already reclaimed the space between the end of bss and the
page end for use by malloc, and memory corruption (allocating the same
memory twice) will occur when malloc again claims it on the first call
to brk.
if there's evidence of any use for it, we can add it back later. as
far as I can tell, glibc has it only for internal use (and musl uses a
direct syscall in that case rather than a function call), not for
exposing it to applications.
in case of mmap-obtained chunks, end points past the end of the
mapping and reading it may fault. since the value is not needed until
after the conditional, move the access to prevent invalid reads.
they were accidentally exposed under just baseline POSIX, which is a
big namespace pollution issue. thankfully glibc only exposes them
under _GNU_SOURCE, not under any of its other options, so omitting
the pollution in the default _BSD_SOURCE profile does not hurt
application compatibility at all.
previously the names were exposed as key/seq with _GNU_SOURCE and
__ipc_perm_key/__ipc_perm/seq otherwise, whereas glibc always uses
__key and __seq for the names. thus, the old behavior never matched
glibc, and the new behavior always does, regardless of feature test
macros.
for now, i'm leaving the renaming here in sys/ipc.h where it's easy to
change globally for all archs, in case something turns out to be
wrong, but eventually the names could just be incorporated directly
into the bits headers for each arch and the renaming removed.
the issue is identical to the recent commit fixing the mips versions:
despite other implementations doing this, it conflicts with the
requirements of ISO C and it's a waste of time and code size.
previously, everything was going through an intermediate conversion to
long double, which caused the extern __fpclassifyl function to get
invoked, preventing virtually all optimizations of these operations.
with the new code, tests on constant float or double arguments compile
to a constant 0 or 1, and tests on non-constant expressions are
efficient. I may later add support for __builtin versions on compilers
that support them.
nothing in the standard requires or even allows the fenv state to be
restored by longjmp. restoring the exception flags is not such a big
deal since it's probably valid to clobber them completely, but
restoring the rounding mode yields an observable side effect not
sanctioned by ISO C. saving/restoring it also wastes a few cycles and
16 bytes of code.
as for historical behavior, reportedly SGI IRIX did save/restore fenv,
and this is where glibc and uClibc got the behavior from. a few other
systems save/restore it too (on archs other than mips), even though
this is apparently wrong. further details are documented here:
http://www-personal.umich.edu/~williams/archive/computation/setjmp-fpmode.html
as musl aims for standards conformance rather than coddling historical
programs expecting non-conforming behavior, and as it's unlikely that
any historical programs actually depend on the incorrect behavior
(such programs would break on other archs, anyway), I'm making the
change not to save/restore fenv on mips.
due to some historical oddity, these are considered libc headers
rather than kernel headers. the kernel used to provide them too, but
it seems modern kernels do not install them, so let's just do the
easiest thing and provide them. stripped-down versions provided by
John Spencer.
apparently recent gcc versions have intentionally broken the
traditional definition by treating it as a non-constant expression.
the traditional definition may also be problematic for c++ programs.
for some reason I have not been able to determine, gcc 3.2 rejects the
array notation. this seems to be a gcc bug, but since it's easy to
work around, let's do the workaround and avoid gratuitously requiring
newer compilers.
previously, a few BSD features were enabled only by _BSD_SOURCE, not
by _GNU_SOURCE. since _BSD_SOURCE is default in the absence of other
feature test macros, this made adding _GNU_SOURCE to a project not a
purely additive feature test macro; it actually caused some features
to be suppressed.
most of the changes made by this patch actually bring musl in closer
alignment with the glibc behavior for _GNU_SOURCE. the only exceptions
are the added visibility of functions like strlcpy which were BSD-only
due to being disliked/rejected by glibc maintainers. here, I feel the
consistency of having _GNU_SOURCE mean "everything", and especially
the property of it being purely additive, are more valuable than
hiding functions which glibc does not have.
most importantly, the format/scan macros for the [u]int_fast16_t and
[u]int_fast32_t types were defined incorrectly assuming these types
would match the native word/pointer size. this is incorrect on any
64-bit system; the "fast" types for 16- and 32-bit integers are simply
int.
another issue which was "only a warning" (despite being UB) is that
the choice of "l" versus "ll" was incorrect for 64-bit types on 64-bit
machines. while it would "work" to always use "long long" for 64-bit
types, we use "long" on 64-bit machines to match what glibc does and
what the ABI documents recommend. the macro definitions were probably
right in very old versions of musl, but became wrong when we aligned
most closely with the 'standard' ABI. checking UINTPTR_MAX is an easy
way to get the system wordsize without pulling in new headers.
finally, the useless __PRIPTR macro to allow the underlying type of
[u]intptr_t to vary has been removed. we are using "long" on all
targets, and thankfully this matches what glibc does, so I do not
envision ever needing to change it. thus, the "l" has just been
incorporated directly in the strings.
previously, shared library constructors were being called before
important internal things like the environment (extern char **environ)
and hwcap flags (needed for sjlj to work right with float on arm) were
initialized in __libc_start_main. rather than trying to have to
dynamic linker make sure this stuff all gets initialized right, I've
opted to just defer calling shared library constructors until after
the main program's entry point is reached. this also fixes the order
of ctors to be the exact reverse of dtors, which is a desirable
property and possibly even mandated by some languages.
the main practical effect of this change is that shared libraries
calling getenv from ctors will no longer fail.
the missing check did not affect the default profile, since it has
both _XOPEN_SOURCE and _BSD_SOURCE defined, but it did break programs
which explicitly define _BSD_SOURCE, causing it to be the only feature
test macro present.
these structures are purely for use by trace/debug tools and tools
working with core files. the definition of fpregset_t, which was
previously here, has been removed because it was wrong; fpregset_t
should be the type used in mcontext_t, not the type used in
ptrace/core stuff.
aside from microblaze, these should be roughly correct for all archs
now. some misc junk macros and typedefs are missing, which should
probably be added for max compatibility with trace/debug tools.
it should now really match the kernel. some of the removed padding
corresponded to the difference between user and kernel sigset_t. the
space at the end was redundant with the uc_mcontext member and seems
to have been added as a result of misunderstanding glibc's definition
versus the kernel's.
Rich Felker