|
perlxs - XS language reference manual
XS is an interface description file format used to create an extension
interface between Perl and C code (or a C library) which one wishes
to use with Perl. The XS interface is combined with the library to
create a new library which can then be either dynamically loaded
or statically linked into perl. The XS interface description is
written in the XS language and is the core component of the Perl
extension interface.
An XSUB forms the basic unit of the XS interface. After compilation
by the xsubpp compiler, each XSUB amounts to a C function definition
which will provide the glue between Perl calling conventions and C
calling conventions.
The glue code pulls the arguments from the Perl stack, converts these
Perl values to the formats expected by a C function, call this C function,
transfers the return values of the C function back to Perl.
Return values here may be a conventional C return value or any C
function arguments that may serve as output parameters. These return
values may be passed back to Perl either by putting them on the
Perl stack, or by modifying the arguments supplied from the Perl side.
The above is a somewhat simplified view of what really happens. Since
Perl allows more flexible calling conventions than C, XSUBs may do much
more in practice, such as checking input parameters for validity,
throwing exceptions (or returning undef/empty list) if the return value
from the C function indicates failure, calling different C functions
based on numbers and types of the arguments, providing an object-oriented
interface, etc.
Of course, one could write such glue code directly in C. However, this
would be a tedious task, especially if one needs to write glue for
multiple C functions, and/or one is not familiar enough with the Perl
stack discipline and other such arcana. XS comes to the rescue here:
instead of writing this glue C code in long-hand, one can write
a more concise short-hand description of what should be done by
the glue, and let the XS compiler xsubpp handle the rest.
The XS language allows one to describe the mapping between how the C
routine is used, and how the corresponding Perl routine is used. It
also allows creation of Perl routines which are directly translated to
C code and which are not related to a pre-existing C function. In cases
when the C interface coincides with the Perl interface, the XSUB
declaration is almost identical to a declaration of a C function (in K&R
style). In such circumstances, there is another tool called h2xs
that is able to translate an entire C header file into a corresponding
XS file that will provide glue to the functions/macros described in
the header file.
The XS compiler is called xsubpp. This compiler creates
the constructs necessary to let an XSUB manipulate Perl values, and
creates the glue necessary to let Perl call the XSUB. The compiler
uses typemaps to determine how to map C function parameters
and output values to Perl values and back. The default typemap
(which comes with Perl) handles many common C types. A supplementary
typemap may also be needed to handle any special structures and types
for the library being linked.
A file in XS format starts with a C language section which goes until the
first MODULE = directive. Other XS directives and XSUB definitions
may follow this line. The "language" used in this part of the file
is usually referred to as the XS language. xsubpp recognizes and
skips POD (see the perlpod manpage) in both the C and XS language sections, which
allows the XS file to contain embedded documentation.
See the perlxstut manpage for a tutorial on the whole extension creation process.
Note: For some extensions, Dave Beazley's SWIG system may provide a
significantly more convenient mechanism for creating the extension
glue code. See http://www.swig.org/ for more information.
Many of the examples which follow will concentrate on creating an interface
between Perl and the ONC+ RPC bind library functions. The rpcb_gettime()
function is used to demonstrate many features of the XS language. This
function has two parameters; the first is an input parameter and the second
is an output parameter. The function also returns a status value.
bool_t rpcb_gettime(const char *host, time_t *timep);
From C this function will be called with the following
statements.
bool_t status;
time_t timep;
status = rpcb_gettime( "localhost", &timep );
If an XSUB is created to offer a direct translation between this function
and Perl, then this XSUB will be used from Perl with the following code.
The $status and $timep variables will contain the output of the function.
use RPC;
$status = rpcb_gettime( "localhost", $timep );
The following XS file shows an XS subroutine, or XSUB, which
demonstrates one possible interface to the rpcb_gettime()
function. This XSUB represents a direct translation between
C and Perl and so preserves the interface even from Perl.
This XSUB will be invoked from Perl with the usage shown
above. Note that the first three #include statements, for
EXTERN.h, perl.h, and XSUB.h, will always be present at the
beginning of an XS file. This approach and others will be
expanded later in this document.
MODULE = RPC PACKAGE = RPC
bool_t
rpcb_gettime(host,timep)
char *host
time_t &timep
OUTPUT:
timep
Any extension to Perl, including those containing XSUBs,
should have a Perl module to serve as the bootstrap which
pulls the extension into Perl. This module will export the
extension's functions and variables to the Perl program and
will cause the extension's XSUBs to be linked into Perl.
The following module will be used for most of the examples
in this document and should be used from Perl with the use
command as shown earlier. Perl modules are explained in
more detail later in this document.
package RPC;
require Exporter;
require DynaLoader;
@ISA = qw(Exporter DynaLoader);
@EXPORT = qw( rpcb_gettime );
bootstrap RPC;
1;
Throughout this document a variety of interfaces to the rpcb_gettime()
XSUB will be explored. The XSUBs will take their parameters in different
orders or will take different numbers of parameters. In each case the
XSUB is an abstraction between Perl and the real C rpcb_gettime()
function, and the XSUB must always ensure that the real rpcb_gettime()
function is called with the correct parameters. This abstraction will
allow the programmer to create a more Perl-like interface to the C
function.
The simplest XSUBs consist of 3 parts: a description of the return
value, the name of the XSUB routine and the names of its arguments,
and a description of types or formats of the arguments.
The following XSUB allows a Perl program to access a C library function
called sin(). The XSUB will imitate the C function which takes a single
argument and returns a single value.
double
sin(x)
double x
Optionally, one can merge the description of types and the list of
argument names, rewriting this as
double
sin(double x)
This makes this XSUB look similar to an ANSI C declaration. An optional
semicolon is allowed after the argument list, as in
double
sin(double x);
Parameters with C pointer types can have different semantic: C functions
with similar declarations
bool string_looks_as_a_number(char *s);
bool make_char_uppercase(char *c);
are used in absolutely incompatible manner. Parameters to these functions
could be described xsubpp like this:
char * s
char &c
Both these XS declarations correspond to the char* C type, but they have
different semantics, see The & Unary Operator.
It is convenient to think that the indirection operator
* should be considered as a part of the type and the address operator &
should be considered part of the variable. See The Typemap
for more info about handling qualifiers and unary operators in C types.
The function name and the return type must be placed on
separate lines and should be flush left-adjusted.
INCORRECT CORRECT
double sin(x) double
double x sin(x)
double x
The rest of the function description may be indented or left-adjusted. The
following example shows a function with its body left-adjusted. Most
examples in this document will indent the body for better readability.
CORRECT
double
sin(x)
double x
More complicated XSUBs may contain many other sections. Each section of
an XSUB starts with the corresponding keyword, such as INIT: or CLEANUP:.
However, the first two lines of an XSUB always contain the same data:
descriptions of the return type and the names of the function and its
parameters. Whatever immediately follows these is considered to be
an INPUT: section unless explicitly marked with another keyword.
(See The INPUT: Keyword.)
An XSUB section continues until another section-start keyword is found.
The Perl argument stack is used to store the values which are
sent as parameters to the XSUB and to store the XSUB's
return value(s). In reality all Perl functions (including non-XSUB
ones) keep their values on this stack all the same time, each limited
to its own range of positions on the stack. In this document the
first position on that stack which belongs to the active
function will be referred to as position 0 for that function.
XSUBs refer to their stack arguments with the macro ST(x), where x
refers to a position in this XSUB's part of the stack. Position 0 for that
function would be known to the XSUB as ST(0). The XSUB's incoming
parameters and outgoing return values always begin at ST(0). For many
simple cases the xsubpp compiler will generate the code necessary to
handle the argument stack by embedding code fragments found in the
typemaps. In more complex cases the programmer must supply the code.
The RETVAL variable is a special C variable that is declared automatically
for you. The C type of RETVAL matches the return type of the C library
function. The xsubpp compiler will declare this variable in each XSUB
with non-void return type. By default the generated C function
will use RETVAL to hold the return value of the C library function being
called. In simple cases the value of RETVAL will be placed in ST(0) of
the argument stack where it can be received by Perl as the return value
of the XSUB.
If the XSUB has a return type of void then the compiler will
not declare a RETVAL variable for that function. When using
a PPCODE: section no manipulation of the RETVAL variable is required, the
section may use direct stack manipulation to place output values on the stack.
If PPCODE: directive is not used, void return value should be used
only for subroutines which do not return a value, even if CODE:
directive is used which sets ST(0) explicitly.
Older versions of this document recommended to use void return
value in such cases. It was discovered that this could lead to
segfaults in cases when XSUB was truly void. This practice is
now deprecated, and may be not supported at some future version. Use
the return value SV * in such cases. (Currently xsubpp contains
some heuristic code which tries to disambiguate between "truly-void"
and "old-practice-declared-as-void" functions. Hence your code is at
mercy of this heuristics unless you use SV * as return value.)
When you're using RETVAL to return an SV *, there's some magic
going on behind the scenes that should be mentioned. When you're
manipulating the argument stack using the ST(x) macro, for example,
you usually have to pay special attention to reference counts. (For
more about reference counts, see the perlguts manpage.) To make your life
easier, the typemap file automatically makes RETVAL mortal when
you're returning an SV *. Thus, the following two XSUBs are more
or less equivalent:
void
alpha()
PPCODE:
ST(0) = newSVpv("Hello World",0);
sv_2mortal(ST(0));
XSRETURN(1);
SV *
beta()
CODE:
RETVAL = newSVpv("Hello World",0);
OUTPUT:
RETVAL
This is quite useful as it usually improves readability. While
this works fine for an SV *, it's unfortunately not as easy
to have AV * or HV * as a return value. You should be
able to write:
AV *
array()
CODE:
RETVAL = newAV();
/* do something with RETVAL */
OUTPUT:
RETVAL
But due to an unfixable bug (fixing it would break lots of existing
CPAN modules) in the typemap file, the reference count of the AV *
is not properly decremented. Thus, the above XSUB would leak memory
whenever it is being called. The same problem exists for HV *.
When you're returning an AV * or a HV *, you have make sure
their reference count is decremented by making the AV or HV mortal:
AV *
array()
CODE:
RETVAL = newAV();
sv_2mortal((SV*)RETVAL);
/* do something with RETVAL */
OUTPUT:
RETVAL
And also remember that you don't have to do this for an SV *.
The MODULE keyword is used to start the XS code and to specify the package
of the functions which are being defined. All text preceding the first
MODULE keyword is considered C code and is passed through to the output with
POD stripped, but otherwise untouched. Every XS module will have a
bootstrap function which is used to hook the XSUBs into Perl. The package
name of this bootstrap function will match the value of the last MODULE
statement in the XS source files. The value of MODULE should always remain
constant within the same XS file, though this is not required.
The following example will start the XS code and will place
all functions in a package named RPC.
MODULE = RPC
When functions within an XS source file must be separated into packages
the PACKAGE keyword should be used. This keyword is used with the MODULE
keyword and must follow immediately after it when used.
MODULE = RPC PACKAGE = RPC
[ XS code in package RPC ]
MODULE = RPC PACKAGE = RPCB
[ XS code in package RPCB ]
MODULE = RPC PACKAGE = RPC
[ XS code in package RPC ]
The same package name can be used more than once, allowing for
non-contiguous code. This is useful if you have a stronger ordering
principle than package names.
Although this keyword is optional and in some cases provides redundant
information it should always be used. This keyword will ensure that the
XSUBs appear in the desired package.
The PREFIX keyword designates prefixes which should be
removed from the Perl function names. If the C function is
rpcb_gettime() and the PREFIX value is rpcb_ then Perl will
see this function as gettime().
This keyword should follow the PACKAGE keyword when used.
If PACKAGE is not used then PREFIX should follow the MODULE
keyword.
MODULE = RPC PREFIX = rpc_
MODULE = RPC PACKAGE = RPCB PREFIX = rpcb_
The OUTPUT: keyword indicates that certain function parameters should be
updated (new values made visible to Perl) when the XSUB terminates or that
certain values should be returned to the calling Perl function. For
simple functions which have no CODE: or PPCODE: section,
such as the sin() function above, the RETVAL variable is
automatically designated as an output value. For more complex functions
the xsubpp compiler will need help to determine which variables are output
variables.
This keyword will normally be used to complement the CODE: keyword.
The RETVAL variable is not recognized as an output variable when the
CODE: keyword is present. The OUTPUT: keyword is used in this
situation to tell the compiler that RETVAL really is an output
variable.
The OUTPUT: keyword can also be used to indicate that function parameters
are output variables. This may be necessary when a parameter has been
modified within the function and the programmer would like the update to
be seen by Perl.
bool_t
rpcb_gettime(host,timep)
char *host
time_t &timep
OUTPUT:
timep
The OUTPUT: keyword will also allow an output parameter to
be mapped to a matching piece of code rather than to a
typemap.
bool_t
rpcb_gettime(host,timep)
char *host
time_t &timep
OUTPUT:
timep sv_setnv(ST(1), (double)timep);
xsubpp emits an automatic SvSETMAGIC() for all parameters in the
OUTPUT section of the XSUB, except RETVAL. This is the usually desired
behavior, as it takes care of properly invoking 'set' magic on output
parameters (needed for hash or array element parameters that must be
created if they didn't exist). If for some reason, this behavior is
not desired, the OUTPUT section may contain a SETMAGIC: DISABLE line
to disable it for the remainder of the parameters in the OUTPUT section.
Likewise, SETMAGIC: ENABLE can be used to reenable it for the
remainder of the OUTPUT section. See the perlguts manpage for more details
about 'set' magic.
The NO_OUTPUT can be placed as the first token of the XSUB. This keyword
indicates that while the C subroutine we provide an interface to has
a non-void return type, the return value of this C subroutine should not
be returned from the generated Perl subroutine.
With this keyword present The RETVAL Variable is created, and in the
generated call to the subroutine this variable is assigned to, but the value
of this variable is not going to be used in the auto-generated code.
This keyword makes sense only if RETVAL is going to be accessed by the
user-supplied code. It is especially useful to make a function interface
more Perl-like, especially when the C return value is just an error condition
indicator. For example,
NO_OUTPUT int
delete_file(char *name)
POSTCALL:
if (RETVAL != 0)
croak("Error %d while deleting file '%s'", RETVAL, name);
Here the generated XS function returns nothing on success, and will die()
with a meaningful error message on error.
This keyword is used in more complicated XSUBs which require
special handling for the C function. The RETVAL variable is
still declared, but it will not be returned unless it is specified
in the OUTPUT: section.
The following XSUB is for a C function which requires special handling of
its parameters. The Perl usage is given first.
$status = rpcb_gettime( "localhost", $timep );
The XSUB follows.
bool_t
rpcb_gettime(host,timep)
char *host
time_t timep
CODE:
RETVAL = rpcb_gettime( host, &timep );
OUTPUT:
timep
RETVAL
The INIT: keyword allows initialization to be inserted into the XSUB before
the compiler generates the call to the C function. Unlike the CODE: keyword
above, this keyword does not affect the way the compiler handles RETVAL.
bool_t
rpcb_gettime(host,timep)
char *host
time_t &timep
INIT:
printf("# Host is %s\n", host );
OUTPUT:
timep
Another use for the INIT: section is to check for preconditions before
making a call to the C function:
long long
lldiv(a,b)
long long a
long long b
INIT:
if (a == 0 && b == 0)
XSRETURN_UNDEF;
if (b == 0)
croak("lldiv: cannot divide by 0");
The NO_INIT keyword is used to indicate that a function
parameter is being used only as an output value. The xsubpp
compiler will normally generate code to read the values of
all function parameters from the argument stack and assign
them to C variables upon entry to the function. NO_INIT
will tell the compiler that some parameters will be used for
output rather than for input and that they will be handled
before the function terminates.
The following example shows a variation of the rpcb_gettime() function.
This function uses the timep variable only as an output variable and does
not care about its initial contents.
bool_t
rpcb_gettime(host,timep)
char *host
time_t &timep = NO_INIT
OUTPUT:
timep
C function parameters are normally initialized with their values from
the argument stack (which in turn contains the parameters that were
passed to the XSUB from Perl). The typemaps contain the
code segments which are used to translate the Perl values to
the C parameters. The programmer, however, is allowed to
override the typemaps and supply alternate (or additional)
initialization code. Initialization code starts with the first
=, ; or + on a line in the INPUT: section. The only
exception happens if this ; terminates the line, then this ;
is quietly ignored.
The following code demonstrates how to supply initialization code for
function parameters. The initialization code is eval'd within double
quotes by the compiler before it is added to the output so anything
which should be interpreted literally [mainly $, @, or \\]
must be protected with backslashes. The variables $var, $arg,
and $type can be used as in typemaps.
bool_t
rpcb_gettime(host,timep)
char *host = (char *)SvPV($arg,PL_na);
time_t &timep = 0;
OUTPUT:
timep
This should not be used to supply default values for parameters. One
would normally use this when a function parameter must be processed by
another library function before it can be used. Default parameters are
covered in the next section.
If the initialization begins with =, then it is output in
the declaration for the input variable, replacing the initialization
supplied by the typemap. If the initialization
begins with ; or +, then it is performed after
all of the input variables have been declared. In the ;
case the initialization normally supplied by the typemap is not performed.
For the + case, the declaration for the variable will include the
initialization from the typemap. A global
variable, %v, is available for the truly rare case where
information from one initialization is needed in another
initialization.
Here's a truly obscure example:
bool_t
rpcb_gettime(host,timep)
time_t &timep; /* \$v{timep}=@{[$v{timep}=$arg]} */
char *host + SvOK($v{timep}) ? SvPV($arg,PL_na) : NULL;
OUTPUT:
timep
The construct \$v{timep}=@{[$v{timep}=$arg]} used in the above
example has a two-fold purpose: first, when this line is processed by
xsubpp, the Perl snippet $v{timep}=$arg is evaluated. Second,
the text of the evaluated snippet is output into the generated C file
(inside a C comment)! During the processing of char *host line,
$arg will evaluate to ST(0), and $v{timep} will evaluate to
ST(1).
Default values for XSUB arguments can be specified by placing an
assignment statement in the parameter list. The default value may
be a number, a string or the special string NO_INIT. Defaults should
always be used on the right-most parameters only.
To allow the XSUB for rpcb_gettime() to have a default host
value the parameters to the XSUB could be rearranged. The
XSUB will then call the real rpcb_gettime() function with
the parameters in the correct order. This XSUB can be called
from Perl with either of the following statements:
$status = rpcb_gettime( $timep, $host );
$status = rpcb_gettime( $timep );
The XSUB will look lik | 
