|
perlport - Writing portable Perl
Perl runs on numerous operating systems. While most of them share
much in common, they also have their own unique features.
This document is meant to help you to find out what constitutes portable
Perl code. That way once you make a decision to write portably,
you know where the lines are drawn, and you can stay within them.
There is a tradeoff between taking full advantage of one particular
type of computer and taking advantage of a full range of them.
Naturally, as you broaden your range and become more diverse, the
common factors drop, and you are left with an increasingly smaller
area of common ground in which you can operate to accomplish a
particular task. Thus, when you begin attacking a problem, it is
important to consider under which part of the tradeoff curve you
want to operate. Specifically, you must decide whether it is
important that the task that you are coding have the full generality
of being portable, or whether to just get the job done right now.
This is the hardest choice to be made. The rest is easy, because
Perl provides many choices, whichever way you want to approach your
problem.
Looking at it another way, writing portable code is usually about
willfully limiting your available choices. Naturally, it takes
discipline and sacrifice to do that. The product of portability
and convenience may be a constant. You have been warned.
Be aware of two important points:
- Not all Perl programs have to be portable
-
There is no reason you should not use Perl as a language to glue Unix
tools together, or to prototype a Macintosh application, or to manage the
Windows registry. If it makes no sense to aim for portability for one
reason or another in a given program, then don't bother.
- Nearly all of Perl already is portable
-
Don't be fooled into thinking that it is hard to create portable Perl
code. It isn't. Perl tries its level-best to bridge the gaps between
what's available on different platforms, and all the means available to
use those features. Thus almost all Perl code runs on any machine
without modification. But there are some significant issues in
writing portable code, and this document is entirely about those issues.
Here's the general rule: When you approach a task commonly done
using a whole range of platforms, think about writing portable
code. That way, you don't sacrifice much by way of the implementation
choices you can avail yourself of, and at the same time you can give
your users lots of platform choices. On the other hand, when you have to
take advantage of some unique feature of a particular platform, as is
often the case with systems programming (whether for Unix, Windows,
Mac OS, VMS, etc.), consider writing platform-specific code.
When the code will run on only two or three operating systems, you
may need to consider only the differences of those particular systems.
The important thing is to decide where the code will run and to be
deliberate in your decision.
The material below is separated into three main sections: main issues of
portability (ISSUES), platform-specific issues (PLATFORMS), and
built-in perl functions that behave differently on various ports
(FUNCTION IMPLEMENTATIONS).
This information should not be considered complete; it includes possibly
transient information about idiosyncrasies of some of the ports, almost
all of which are in a state of constant evolution. Thus, this material
should be considered a perpetual work in progress
(<IMG SRC="yellow_sign.gif" ALT="Under Construction">).
In most operating systems, lines in files are terminated by newlines.
Just what is used as a newline may vary from OS to OS. Unix
traditionally uses \012, one type of DOSish I/O uses \015\012,
and Mac OS uses \015.
Perl uses \n to represent the "logical" newline, where what is
logical may depend on the platform in use. In MacPerl, \n always
means \015. In DOSish perls, \n usually means \012, but
when accessing a file in "text" mode, STDIO translates it to (or
from) \015\012, depending on whether you're reading or writing.
Unix does the same thing on ttys in canonical mode. \015\012
is commonly referred to as CRLF.
To trim trailing newlines from text lines use chomp(). With default
settings that function looks for a trailing \n character and thus
trims in a portable way.
When dealing with binary files (or text files in binary mode) be sure
to explicitly set $/ to the appropriate value for your file format
before using chomp().
Because of the "text" mode translation, DOSish perls have limitations
in using seek and tell on a file accessed in "text" mode.
Stick to seek-ing to locations you got from tell (and no
others), and you are usually free to use seek and tell even
in "text" mode. Using seek or tell or other file operations
may be non-portable. If you use binmode on a file, however, you
can usually seek and tell with arbitrary values in safety.
A common misconception in socket programming is that \n eq \012
everywhere. When using protocols such as common Internet protocols,
\012 and \015 are called for specifically, and the values of
the logical \n and \r (carriage return) are not reliable.
print SOCKET "Hi there, client!\r\n";
print SOCKET "Hi there, client!\015\012";
However, using \015\012 (or \cM\cJ, or \x0D\x0A) can be tedious
and unsightly, as well as confusing to those maintaining the code. As
such, the Socket module supplies the Right Thing for those who want it.
use Socket qw(:DEFAULT :crlf);
print SOCKET "Hi there, client!$CRLF"
When reading from a socket, remember that the default input record
separator $/ is \n, but robust socket code will recognize as
either \012 or \015\012 as end of line:
while (<SOCKET>) {
}
Because both CRLF and LF end in LF, the input record separator can
be set to LF and any CR stripped later. Better to write:
use Socket qw(:DEFAULT :crlf);
local($/) = LF;
while (<SOCKET>) {
s/$CR?$LF/\n/;
}
This example is preferred over the previous one--even for Unix
platforms--because now any \015's (\cM's) are stripped out
(and there was much rejoicing).
Similarly, functions that return text data--such as a function that
fetches a web page--should sometimes translate newlines before
returning the data, if they've not yet been translated to the local
newline representation. A single line of code will often suffice:
$data =~ s/\015?\012/\n/g;
return $data;
Some of this may be confusing. Here's a handy reference to the ASCII CR
and LF characters. You can print it out and stick it in your wallet.
LF eq \012 eq \x0A eq \cJ eq chr(10) eq ASCII 10
CR eq \015 eq \x0D eq \cM eq chr(13) eq ASCII 13
| Unix | DOS | Mac |
---------------------------
\n | LF | LF | CR |
\r | CR | CR | LF |
\n * | LF | CRLF | CR |
\r * | CR | CR | LF |
---------------------------
* text-mode STDIO
The Unix column assumes that you are not accessing a serial line
(like a tty) in canonical mode. If you are, then CR on input becomes
"\n", and "\n" on output becomes CRLF.
These are just the most common definitions of \n and \r in Perl.
There may well be others. For example, on an EBCDIC implementation
such as z/OS (OS/390) or OS/400 (using the ILE, the PASE is ASCII-based)
the above material is similar to "Unix" but the code numbers change:
LF eq \025 eq \x15 eq \cU eq chr(21) eq CP-1047 21
LF eq \045 eq \x25 eq chr(37) eq CP-0037 37
CR eq \015 eq \x0D eq \cM eq chr(13) eq CP-1047 13
CR eq \015 eq \x0D eq \cM eq chr(13) eq CP-0037 13
| z/OS | OS/400 |
----------------------
\n | LF | LF |
\r | CR | CR |
\n * | LF | LF |
\r * | CR | CR |
----------------------
* text-mode STDIO
Different CPUs store integers and floating point numbers in different
orders (called endianness) and widths (32-bit and 64-bit being the
most common today). This affects your programs when they attempt to transfer
numbers in binary format from one CPU architecture to another,
usually either "live" via network connection, or by storing the
numbers to secondary storage such as a disk file or tape.
Conflicting storage orders make utter mess out of the numbers. If a
little-endian host (Intel, VAX) stores 0x12345678 (305419896 in
decimal), a big-endian host (Motorola, Sparc, PA) reads it as
0x78563412 (2018915346 in decimal). Alpha and MIPS can be either:
Digital/Compaq used/uses them in little-endian mode; SGI/Cray uses
them in big-endian mode. To avoid this problem in network (socket)
connections use the pack and unpack formats n and N, the
"network" orders. These are guaranteed to be portable.
As of perl 5.8.5, you can also use the > and < modifiers
to force big- or little-endian byte-order. This is useful if you want
to store signed integers or 64-bit integers, for example.
You can explore the endianness of your platform by unpacking a
data structure packed in native format such as:
print unpack("h*", pack("s2", 1, 2)), "\n";
If you need to distinguish between endian architectures you could use
either of the variables set like so:
$is_big_endian = unpack("h*", pack("s", 1)) =~ /01/;
$is_little_endian = unpack("h*", pack("s", 1)) =~ /^1/;
Differing widths can cause truncation even between platforms of equal
endianness. The platform of shorter width loses the upper parts of the
number. There is no good solution for this problem except to avoid
transferring or storing raw binary numbers.
One can circumnavigate both these problems in two ways. Either
transfer and store numbers always in text format, instead of raw
binary, or else consider using modules like Data::Dumper (included in
the standard distribution as of Perl 5.005) and Storable (included as
of perl 5.8). Keeping all data as text significantly simplifies matters.
The v-strings are portable only up to v2147483647 (0x7FFFFFFF), that's
how far EBCDIC, or more precisely UTF-EBCDIC will go.
Most platforms these days structure files in a hierarchical fashion.
So, it is reasonably safe to assume that all platforms support the
notion of a "path" to uniquely identify a file on the system. How
that path is really written, though, differs considerably.
Although similar, file path specifications differ between Unix,
Windows, Mac OS, OS/2, VMS, VOS, RISC OS, and probably others.
Unix, for example, is one of the few OSes that has the elegant idea
of a single root directory.
DOS, OS/2, VMS, VOS, and Windows can work similarly to Unix with /
as path separator, or in their own idiosyncratic ways (such as having
several root directories and various "unrooted" device files such NIL:
and LPT:).
Mac OS uses : as a path separator instead of /.
The filesystem may support neither hard links (link) nor
symbolic links (symlink, readlink, lstat).
The filesystem may support neither access timestamp nor change
timestamp (meaning that about the only portable timestamp is the
modification timestamp), or one second granularity of any timestamps
(e.g. the FAT filesystem limits the time granularity to two seconds).
The "inode change timestamp" (the -C filetest) may really be the
"creation timestamp" (which it is not in UNIX).
VOS perl can emulate Unix filenames with / as path separator. The
native pathname characters greater-than, less-than, number-sign, and
percent-sign are always accepted.
RISC OS perl can emulate Unix filenames with / as path
separator, or go native and use . for path separator and : to
signal filesystems and disk names.
Don't assume UNIX filesystem access semantics: that read, write,
and execute are all the permissions there are, and even if they exist,
that their semantics (for example what do r, w, and x mean on
a directory) are the UNIX ones. The various UNIX/POSIX compatibility
layers usually try to make interfaces like chmod() work, but sometimes
there simply is no good mapping.
If all this is intimidating, have no (well, maybe only a little)
fear. There are modules that can help. The File::Spec modules
provide methods to do the Right Thing on whatever platform happens
to be running the program.
use File::Spec::Functions;
chdir(updir());
$file = catfile(curdir(), 'temp', 'file.txt');
File::Spec is available in the standard distribution as of version
5.004_05. File::Spec::Functions is only in File::Spec 0.7 and later,
and some versions of perl come with version 0.6. If File::Spec
is not updated to 0.7 or later, you must use the object-oriented
interface from File::Spec (or upgrade File::Spec).
In general, production code should not have file paths hardcoded.
Making them user-supplied or read from a configuration file is
better, keeping in mind that file path syntax varies on different
machines.
This is especially noticeable in scripts like Makefiles and test suites,
which often assume / as a path separator for subdirectories.
Also of use is File::Basename from the standard distribution, which
splits a pathname into pieces (base filename, full path to directory,
and file suffix).
Even when on a single platform (if you can call Unix a single platform),
remember not to count on the existence or the contents of particular
system-specific files or directories, like /etc/passwd,
/etc/sendmail.conf, /etc/resolv.conf, or even /tmp/. For
example, /etc/passwd may exist but not contain the encrypted
passwords, because the system is using some form of enhanced security.
Or it may not contain all the accounts, because the system is using NIS.
If code does need to rely on such a file, include a description of the
file and its format in the code's documentation, then make it easy for
the user to override the default location of the file.
Don't assume a text file will end with a newline. They should,
but people forget.
Do not have two files or directories of the same name with different
case, like test.pl and Test.pl, as many platforms have
case-insensitive (or at least case-forgiving) filenames. Also, try
not to have non-word characters (except for .) in the names, and
keep them to the 8.3 convention, for maximum portability, onerous a
burden though this may appear.
Likewise, when using the AutoSplit module, try to keep your functions to
8.3 naming and case-insensitive conventions; or, at the least,
make it so the resulting files have a unique (case-insensitively)
first 8 characters.
Whitespace in filenames is tolerated on most systems, but not all,
and even on systems where it might be tolerated, some utilities
might become confused by such whitespace.
Many systems (DOS, VMS ODS-2) cannot have more than one . in their
filenames.
Don't assume > won't be the first character of a filename.
Always use < explicitly to open a file for reading, or even
better, use the three-arg version of open, unless you want the user to
be able to specify a pipe open.
open(FILE, '<', $existing_file) or die $!;
If filenames might use strange characters, it is safest to open it
with sysopen instead of open. open is magic and can
translate characters like >, <, and |, which may
be the wrong thing to do. (Sometimes, though, it's the right thing.)
Three-arg open can also help protect against this translation in cases
where it is undesirable.
Don't use : as a part of a filename since many systems use that for
their own semantics (Mac OS Classic for separating pathname components,
many networking schemes and utilities for separating the nodename and
the pathname, and so on). For the same reasons, avoid @, ; and
|.
Don't assume that in pathnames you can collapse two leading slashes
// into one: some networking and clustering filesystems have special
semantics for that. Let the operating system to sort it out.
The portable filename characters as defined by ANSI C are
a b c d e f g h i j k l m n o p q r t u v w x y z
A B C D E F G H I J K L M N O P Q R T U V W X Y Z
0 1 2 3 4 5 6 7 8 9
. _ -
and the "-" shouldn't be the first character. If you want to be
hypercorrect, stay case-insensitive and within the 8.3 naming
convention (all the files and directories have to be unique within one
directory if their names are lowercased and truncated to eight
characters before the ., if any, and to three characters after the
., if any). (And do not use .s in directory names.)
Not all platforms provide a command line. These are usually platforms
that rely primarily on a Graphical User Interface (GUI) for user
interaction. A program requiring a command line interface might
not work everywhere. This is probably for the user of the program
to deal with, so don't stay up late worrying about it.
Some platforms can't delete or rename files held open by the system,
this limitation may also apply to changing filesystem metainformation
like file permissions or owners. Remember to close files when you
are done with them. Don't unlink or rename an open file. Don't
tie or open a file already tied or opened; untie or close
it first.
Don't open the same file more than once at a time for writing, as some
operating systems put mandatory locks on such files.
Don't assume that write/modify permission on a directory gives the
right to add or delete files/directories in that directory. That is
filesystem specific: in some filesystems you need write/modify
permission also (or even just) in the file/directory itself. In some
filesystems (AFS, DFS) the permission to add/delete directory entries
is a completely separate permission.
Don't assume that a single unlink completely gets rid of the file:
some filesystems (most notably the ones in VMS) have versioned
filesystems, and unlink() removes only the most recent one (it doesn't
remove all the versions because by default the native tools on those
platforms remove just the most recent version, too). The portable
idiom to remove all the versions of a file is
1 while unlink "file";
This will terminate if the file is undeleteable for some reason
(protected, not there, and so on).
Don't count on a specific environment variable existing in %ENV.
Don't count on %ENV entries being case-sensitive, or even
case-preserving. Don't try to clear %ENV by saying %ENV = ();, or,
if you really have to, make it conditional on $^O ne 'VMS' since in
VMS the %ENV table is much more than a per-process key-value string
table.
Don't count on signals or %SIG for anything.
Don't count on filename globbing. Use opendir, readdir, and
closedir instead.
Don't count on per-program environment variables, or per-program current
directories.
Don't count on specific values of $!, neither numeric nor
especially the strings values-- users may switch their locales causing
error messages to be translated into their languages. If you can
trust a POSIXish environment, you can portably use the symbols defined
by the Errno module, like ENOENT. And don't trust on the values of $!
at all except immediately after a failed system call.
Don't assume that the name used to invoke a command or program with
system or exec can also be used to test for the existence of the
file that holds the executable code for that command or program.
First, many systems have "internal" commands that are built-in to the
shell or OS and while these commands can be invoked, there is no
corresponding file. Second, some operating systems (e.g., Cygwin,
DJGPP, OS/2, and VOS) have required suffixes for executable files;
these suffixes are generally permitted on the command name but are not
required. Thus, a command like "perl" might exist in a file named
"perl", "perl.exe", or "perl.pm", depending on the operating system.
The variable "_exe" in the Config module holds the executable suffix,
if any. Third, the VMS port carefully sets up $^X and
$Config{perlpath} so that no further processing is required. This is
just as well, because the matching regular expression used below would
then have to deal with a possible trailing version number in the VMS
file name.
To convert $^X to a file pathname, taking account of the requirements
of the various operating system possibilities, say:
use Config;
$thisperl = $^X;
if ($^O ne 'VMS')
{$thisperl .= $Config{_exe} unless $thisperl =~ m/$Config{_exe}$/i;}
To convert $Config{perlpath} to a file pathname, say:
use Config;
$thisperl = $Config{perlpath};
if ($^O ne 'VMS')
{$thisperl .= $Config{_exe} unless $thisperl =~ m/$Config{_exe}$/i;}
Don't assume that you can reach the public Internet.
Don't assume that there is only one way to get through firewalls
to the public Internet.
Don't assume that you can reach outside world through any other port
than 80, or some web proxy. ftp is blocked by many firewalls.
Don't assume that you can send email by connecting to the local SMTP port.
Don't assume that you can reach yourself or any node by the name
'localhost'. The same goes for '127.0.0.1'. You will have to try both.
Don't assume that the host has only one network card, or that it
can't bind to many virtual IP addresses.
Don't assume a particular network device name.
Don't assume a particular set of ioctl()s will work.
Don't assume that you can ping hosts and get replies.
Don't assume that any particular port (service) will respond.
Don't assume that Sys::Hostname (or any other API or command)
returns either a fully qualified hostname or a non-qualified hostname:
it all depends on how the system had been configured. Also remember
things like DHCP and NAT-- the hostname you get back might not be very
useful.
All the above "don't":s may look daunting, and they are -- but the key
is to degrade gracefully if one cannot reach the particular network
service one wants. Croaking or hanging do not look very professional.
In general, don't directly access the system in code meant to be
portable. That means, no system, exec, fork, pipe,
``, qx//, open with a |, nor any of the other things
that makes being a perl hacker worth being.
Commands that launch external processes are generally supported on
most platforms (though many of them do not support any type of
forking). The problem with using them arises from what you invoke
them on. External tools are often named differently on different
platforms, may not be available in the same location, might accept
different arguments, can behave differently, and often present their
results in a platform-dependent way. Thus, you should seldom depend
on them to produce consistent results. (Then again, if you're calling
netstat -a, you probably don't expect it to run on both Unix and CP/M.)
One especially common bit of Perl code is opening a pipe to sendmail:
open(MAIL, '|/usr/lib/sendmail -t')
or die "cannot fork sendmail: $!";
This is fine for systems programming when sendmail is known to be
available. But it is not fine for many non-Unix systems, and even
some Unix systems that may not have sendmail installed. If a portable
solution is needed, see the various distributions on CPAN that deal
with it. Mail::Mailer and Mail::Send in the MailTools distribution are
commonly used, and provide several mailing methods, including mail,
sendmail, and direct SMTP (via Net::SMTP) if a mail transfer agent is
not available. Mail::Sendmail is a standalone module that provides
simple, platform-independent mailing.
The Unix System V IPC (msg*(), sem*(), shm*()) is not available
even on all Unix platforms.
Do not use either the bare result of pack("N", 10, 20, 30, 40) or
bare v-strings (such as v10.20.30.40) to represent IPv4 addresses:
both forms just pack the four bytes into network order. That this
would be equal to the C language in_addr struct (which is what the
socket code internally uses) is not guaranteed. To be portable use
the routines of the Socket extension, such as inet_aton(),
inet_ntoa(), and sockaddr_in().
The rule of thumb for portable code is: Do it all in portable Perl, or
use a module (that may internally implement it with platform-specific
code, but expose a common interface).
XS code can usually be made to work with any platform, but dependent
libraries, header files, etc., might not be readily available or
portable, or the XS code itself might be platform-specific, just as Perl
code might be. If the libraries and headers are portable, then it is
normally reasonable to make sure the XS code is portable, too.
A different type of portability issue arises when writing XS code:
availability of a C compiler on the end-user's system. C brings
with it its own portability issues, and writing XS code will expose
you to some of those. Writing purely in Perl is an easier way to
achieve portability.
In general, the standard modules work across platforms. Notable
exceptions are the CPAN module (which currently makes connections to external
programs that may not be available), platform-specific modules (like
ExtUtils::MM_VMS), and DBM modules.
There is no one DBM module available on all platforms.
SDBM_File and the others are generally available on all Unix and DOSish
ports, but not in MacPerl, where only NBDM_File and DB_File are
available.
The good news is that at least some DBM module should be available, and
AnyDBM_File will use whichever module it can find. Of course, then
the code needs to be fairly strict, dropping to the greatest common
factor (e.g., not exceeding 1K for each record), so that it will
work with any DBM module. See the AnyDBM_File manpage for more details.
The system's notion of time of day and calendar date is controlled in
widely different ways. Don't assume the timezone is stored in $ENV{TZ},
and even if it is, don't assume that you can control the timezone through
that variable. Don't assume anything about the three-letter timezone
abbreviations (for example that MST would be the Mountain Standard Time,
it's been known to stand for Moscow Standard Time). If you need to
use timezones, express them in some unambiguous format like the
exact number of minutes offset from UTC, or the POSIX timezone
format.
Don't assume that the epoch starts at 00:00:00, January 1, 1970,
because that is OS- and implementation-specific. It is better to
store a date in an unambiguous representation. The ISO 8601 standard
defines YYYY-MM-DD as the date format, or YYYY-MM-DDTHH-MM-SS
(that's a literal "T" separating the date from the time).
Please do use the ISO 8601 instead of making us to guess what
date 02/03/04 might be. ISO 8601 even sorts nicely as-is.
A text representation (like "1987-12-18") can be easily converted
into an OS-specific value using a module like Date::Parse.
An array of values, such as those returned by localtime, can be
converted to an OS-specific representation using Time::Local.
When calculating specific times, such as for tests in time or date modules,
it may be appropriate to calculate an offset for the epoch.
require Time::Local;
$offset =</ | 
