D. J. Bernstein
UNIX
UNIX portability notes
This web page describes various differences
between FreeBSD, Linux, Solaris, etc.
from a programmer's perspective.
Some of the differences are for good reasons
(for example, a new feature has been added to one system,
and the others haven't yet caught up);
others are for frivolous reasons
(for example, a distributor was too lazy to protect its include files).
Some of the differences are particularly dangerous
because they don't cause obvious compilation failures.
Many portability problems would disappear
if UNIX distributors followed the rule stated by Kernighan and Pike
in The Practice of Programming, Section 8.7:
``Change the name if you change the specification.''
The original author of a program can often get away with spec changes,
because everyone is going to upgrade eventually;
but spec changes made by distributors
are often ignored by other distributors.
For example,
signal() has been a portability disaster for twenty years,
because some distributors (starting with CSRG)
decided to change the signal() spec without changing the name,
while other distributors decided to leave the spec alone.
Notes to programmers:
- Using a ``standard'' function to do something simple?
Consider the possibility that the function isn't portable.
Consider writing a replacement function instead.
- Using cpp to check for machine features?
Consider the possibility that the relevant cpp macros aren't portable.
Consider writing a test program that tests for the features instead.
For example,
if you're trying to figure out which types a C compiler supports
(e.g., to provide maximum alignment to a union),
don't use #ifdefs of macros supposedly related to the types;
simply try compiling each type.
- Using command-line options to modify a program's output?
Consider the possibility that the options aren't portable.
Consider using a pipe to a separate program that modifies the program's output.
accept
On some systems (Linux 2.0; Linux 2.2; HP-UX 10.20),
O_NONBLOCK is not copied from a listening socket
to an accepted socket.
Correct behavior (BSD; Solaris)
is to copy O_NONBLOCK.
Workaround: Set O_NONBLOCK manually.
awk
Sun ships both awk and nawk.
Some vendors ship nawk but call it awk.
Some vendors ship gawk, calling it awk, nawk, and gawk.
awk and nawk have different interpretations of commas after sprintf;
awk treats the string 3x as the number 0, not 3;
awk doesn't support sub, gsub, gensub, delete, tolower, et al.
bash
Old versions of bash don't support ((i++)).
Fix: i=`expr $i + 1`.
char
Reported by Seth Kurtzberg:
By default, char is unsigned char with the AIX compiler.
Some linkers don't give generic alignment to huge byte arrays.
Compiler bug:
gcc 2.8.* and 2.95 fold string literals
if the string literals have the same length
and the same bytes through the first \0.
For example,
(x ? "\1\0\2" : "\1\0\3") is folded to "\1\0\2".
chmod
Under BSD/OS, the sticky bit can't be set on normal files.
connect
Under OpenBSD, connect() to 0.0.0.0 fails.
Other systems connect to the primary interface.
Portably determining the error after a non-blocking connection is tricky.
I have a
separate web page discussing this.
date
Reported by Peter Johnson:
``On NetBSD and FreeBSD, the command 'date -r sec'
prints out the date and time "sec" seconds since the Epoch.
On Redhat and Debian, 'date -r file'
prints out the date and time when file "file" was last modified.''
diff
HP-UX diff doesn't support -a or -q. (Also -d?)
Kernel designers often add
system-specific macros to errno.h.
Workaround:
in files that include errno.h,
avoid all macros beginning with the letter E.
Recent Linux distributions (recent versions of glibc)
refuse to compile programs that declare
extern int errno without including errno.h.
This is a bug, as discussed below.
Correct behavior
(all other C libraries: BSD, Solaris, etc.)
is to accept extern int errno whether or not errno.h
is included.
The workaround for this Linux bug is to include errno.h everywhere,
for example by compiling with gcc -include /usr/include/errno.h.
Further comments on the Linux bug:
When this change was introduced,
it was a clear and flagrant violation of IEEE Std 1003.1-1990 (POSIX),
which specifies the correct declaration to be extern int errno,
nothing more, nothing less.
When Paul Wanish formally requested in 2001 that IEEE issue an "interpretation"
of IEEE Std 1003.1-1990 as meaning something other than what it says,
IEEE refused:
IEEE Std 1003.1-1990 specifies that
extern int errno;
is the correct declaration for errno. This is not a shorthand
for a different meaning.
The glibc authors' excuse for violating IEEE Std 1003.1-1990
(and for loudly demanding changes in subsequent standards)
is that threaded programs need a variable errno address.
This excuse doesn't stand up to even the slightest examination.
Yes, threaded programs need to include errno.h,
but it's trivial to keep extern int errno working
for all the non-threaded programs.
The C library simply defines an int errno
and has its address as the initial value for the thread errno pointer.
expr
expr match x y is Linux-specific.
expr x : y is portable.
expr index xyz abc is Linux-specific.
echo xyz | sed -n 's/[abc].*//p' | wc -c is portable.
find
Old versions of find allow
find -perm -mmm to look for files
whose mode bits include mmm,
and
find -perm mmm to look for files
whose mode bits are exactly mmm,
but not find -perm +mmm to look for files
whose mode bits include at least one of mmm,
For example,
they support
find -perm -0700
to look for executables,
but not
find -perm +0700.
Old versions of find don't allow -maxdepth.
Old versions of find don't allow -L.
Workaround:
replace, e.g.,
find -L srctree -name '*.c'
with
find srctree -follow -name '*.c'.
The Linux manual pages say that -follow is deprecated;
it would be easier to take this seriously if -L were portable.
The BusyBox version of find doesn't allow exec.
Workaround:
replace, e.g.,
find top -type f -exec inspect '{}' ';'
with
find top -type f -print0 | xargs -n1 -0 inspect.
But, oops, the BusyBox version of xargs doesn't allow -0.
Sigh.
flock
sys/file.h is not protected (i.e., can't be included twice) under SCO.
HP/UX and Solaris don't support flock().
On some systems, read-only files can't be locked.
(This is a good thing.)
gethostname
Under Solaris, gethostname() needs -lsocket -lnsl.
On some sysctl-based systems,
gethostname() doesn't write anything if the output buffer is too small.
It should write a truncated name.
getpwnam
On normal systems,
getpwnam() returns passwords if it's running as root.
On Linux,
getpwnam() returns passwords only for users without shadow passwords;
one must also try the non-portable getspnam()
and use its results if it succeeds.
Similarly, on AIX,
one must try the non-portable getuserpw().
PAM-based systems can put 0 into pw_passwd.
getrusage
FreeBSD and OpenBSD need sys/time.h, not just time.h, before sys/resource.h.
grep
Solaris grep doesn't support -q.
Solution: change grep -q to grep 2>/dev/null.
Solaris grep doesn't support -E.
Solution: change grep -E to egrep.
groups
Under Solaris, /usr/bin/groups reports the group in /etc/group,
rather than the results of getgroups().
hostname
Unisys SVR4 doesn't have the hostname command in the standard user path.
inittab
Under HP/UX, Solaris, et al.,
commands run by init are subject to job-control signals.
lockf
HP/UX reportedly returns EACCES for lock failures.
Solaris, HP/UX, et al.
allow one process to have many exclusive locks on one file.
logger
On some systems,
logger uses syslog(pri,buf) instead of syslog(pri,"%s",buf),
so it could barf or crash if fed messages containing %.
long long
Compiler bug:
gcc 2.95.2 -O2 massively screws up long long parameter passing.
-pedantic breaks long long under BSD.
mail
Under RISC/OS, BSD mail passes the -bm option to sendmail.
Under NEWS-OS, BSD mail passes the -E and -J options to sendmail.
make
Some versions of make
don't understand that a line with just a tab is blank.
md5sum
BSD has md5 (and openssl md5), not md5sum.
Linux has md5sum (and sometimes openssl md5), not md5.
Workaround: Pipe input through
( openssl md5 || md5sum || md5 )
to use whichever one works.
Solaris doesn't have any of these.
Workaround, if you simply want some checksum,
not consistent across systems:
( openssl md5 || md5sum || md5 || cksum )
mkdir
Some systems (examples: Linux, OpenBSD)
return EISDIR for mkdir("/",...).
Correct behavior (example: Solaris)
is to return EEXIST.
mkfifo
Apparently no longer a problem
(although the old mknod seems fine too).
I haven't seen any -mkfifo reports since August 1999.
O_NDELAY
Solaris 2.5.1 incorrectly converts O_NDELAY into O_NONBLOCK for sockets.
The result is that clearing O_NDELAY doesn't undo setting O_NDELAY.
Workaround: Use O_NONBLOCK instead.
Nowadays it seems that O_NONBLOCK is perfectly portable,
so there's no reason to use O_NDELAY.
poll
Darwin (MacOS X) still doesn't support poll().
Library bug:
The RedHat 5.1 poll() emulation library
doesn't clear revents when select() returns 0.
Workaround (e.g., in my io library):
Set revents to 0 on entry.
Kernel bug:
On FreeBSD 4.8,
poll() on a bad descriptor fails to indicate POLLIN and POLLOUT;
it indicates only POLLERR.
The idea of handling descriptors separately
(rather than exploding the entire poll with an EBADF) is good,
but there's no excuse for not indicating readability and writability.
Workaround: Check for POLLERR along with POLLIN and POLLOUT.
Kernel bug:
On Linux and Solaris,
poll() fails to set POLLIN for EOF on some types of files, notably pipes.
(Richard Kettlewell has a detailed comparison chart
showing which systems return which combinations of POLLIN and POLLHUP.)
Workaround: Check for POLLHUP along with POLLIN.
_POSIX_VDISABLE
SunOS doesn't have _POSIX_VDISABLE.
(But it does have _PC_VDISABLE.)
OSF/1 has _POSIX_VDISABLE in unistd.h rather than termios.h.
ptys
Under OSF/1, sys/ptms.h is spelled sys/ptys.h.
Kernel bug:
On BSD/OS 2, FreeBSD 4.8, and probably other systems,
a single EOF typed on a tty in the middle of a cooked line
will generate a 0-length packet
if the previous read() on the line had a buffer length
exactly matching the length of the previous packet.
ranlib
Systems returning the following uname results don't have ranlib:
sunos-5.*, unix?sv*, irix64-*, irix-*, dgux-*, hp-ux-*, sco*.
Matthew Dempsky points out a bug in ranlib on MacOS X 10.4.10 (and maybe more versions),
and three workarounds.
Bug:
ranlib forgets about implicitly initialized global variables,
rather than correctly initializing them to 0.
Workaround 1:
Use the -c option to ranlib.
Workaround 2:
Explicitly initialize all global variables.
Workaround 3:
Compile with -fno-common.
readdir
NEWS-OS needs sys/types.h before dirent.h.
res_query
NCR needs stdio.h before arpa/nameser.h.
Workaround: Use the
client library from djbdns.
route
Not all systems have net/route.h.
scanf
Old systems (for example, FreeBSD 4.8) don't understand %lld; they need %qd.
select
Kernel bug:
On some systems,
select() on an almost-full pipe says writable.
(Some systems with the bug in 1996:
AIX 3.2; some HP/UX; Linux 1.2.13; SunOS 4.1.4.)
It should make sure that there are PIPE_BUF bytes available.
(Some systems that did the right thing in 1996: BSD/OS 2.0.1; SunOS 5.4.)
Workaround (e.g., in my io library):
Don't trust positive responses from select();
even if select() has said writable and you're the only writer, go through
all necessary contortions
to make sure that you don't block.
Kernel bug:
On Solaris (through 2.7, I believe),
select() may falsely indicate readability of a socket.
Workaround:
Don't trust positive responses from select().
On several systems,
select() by a fifo reader returns 1
if there aren't any writers yet.
Under Linux,
the fifo reader doesn't wake up if the fifo writer
closes without writing anything;
but Laurent Bercot reports that this has been fixed.
setgroups
HP/UX 9 doesn't allow setgroups() to 65537.
HP/UX 10 and Solaris 2.5 don't allow setgroups()/setgid()
to the system's nogroups/nobody gid.
Some systems don't include the gid in the supplementary group list.
Supplementary groups are int on some systems where gid_t is short.
sh
The behavior of -e varies across systems.
For example,
sh -ec '(exit 1); echo x'
sh -ec '(false; echo y); echo x'
prints nothing under Solaris and FreeBSD
but prints x x under Linux.
Drake Wilson says that x x is mandated by the current POSIX specification.
shutdown
Under Solaris, shutdown() needs -lsocket -lnsl.
sigaction
Apparently no longer a problem.
I haven't seen any -sigaction reports since August 1999.
SIGEMT
Linux doesn't have SIGEMT.
SIGSYS
Linux doesn't have SIGSYS.
SIOCGIFCONF
On some versions of FreeBSD,
SIOCGIFCONF can write past the end of the user-supplied buffer.
(Fixed in February 2000.)
UnixWare returns >0 rather than 0.
BSD 4.4 truncates long lists---and returns success!
There are two incompatible SIOCGIFCONF interfaces.
Workaround: See my ipme library;
or rethink why you're using SIOCGIFCONF in the first place.
socketpair
Under Linux, when one end of a SOCK_DGRAM socketpair is closed,
the other end hangs in read(), recv(), select(), etc.
Correct behavior (BSD)
is to provide an immediate response
from read(), recv(), select(), etc.
sqrt
Sun's cc recognizes sqrt() as a builtin,
even if math.h is not included and sqrt() is defined statically.
syslog
Under SCO OSR5, syslog() and openlog() need -lsocket -lnsl.
Under ESIX, syslog() and openlog() need -lgen.
Workaround: Ever heard of stderr?
tail
Solaris tail doesn't support -n.
Solution: replace tail -n 1 with tail -1.
tcsetpgrp
Under OSF/1, tcsetpgrp() on an unreadable fd fails with EACCES.
time_t
NEWS-OS needs sys/types.h, not just time.h, for time_t.
On AIX, sys/time.h doesn't include time.h;
in particular, it doesn't define struct tm.
At some point the glibc developers
deliberately added the same problem to glibc.
tr
On Solaris 5.8,
tr -cd '[a-z]' deletes all characters except abcdefghijklmnopqrstuvwxyz
while
tr -cd 'a-z' deletes all characters except a, z, hyphen.
On Linux,
tr -cd '[a-z]' deletes all characters except abcdefghijklmnopqrstuvwxyz and brackets,
while
tr -cd 'a-z' deletes all characters except abcdefghijklmnopqrstuvwxyz.
vfork
Irix doesn't have vfork().
vi
Reported by Seth Kurtzberg:
``The command in vi that exchanges two characters is "fixed" in Aix.
One uses the characters to the left and right of the cursor,
and the other uses two characters to the right of the cursor.''
waitpid
Apparently no longer a problem.
I haven't seen any -waitpid reports since January 2000.