Hi Christian,

I took a look into your patch and found it too difficult.
So I implemented another patch (attached) which is based on your ideas.

From user's level of view it does exactly the same except for "lexical"
variables definition. I don't use any new reserver word because every
new reserved word is going to break some user code. I use the special
syntax for lambda function definition instead, which looks much clear
for me. The following code creates a lambda function with arguments $x,
$y and lexical variables $a, $b, $c.

$a = function($x, $y | $a, $b $c) {};

The patch shouldn't affect opcode caches and other extensions as it
doesn't change any structures. It uses the op_array-> static_variables
for lexical variables.

The patch also fixes several small issues and adds some missing
functionality which didn't allow preg_replace_callback() (and may be
others) to work with lambda functions. Now the following example works fine.

<?php
class X {
  private function foo($x) {
    echo $x;
  }
  function bar($s) {
    return function ($x | $s) {
      static $n = 0;
      $n++;
      $s = $n.':'.$s;
      $this-> foo($x[0].':'.$s);
    };
  }
}

$x = new X;
$x = $x-> bar("bye\n");
$s = 'abc';
preg_replace_callback('/[abc]/', $x, $s);
?> 

It prints:

a:1:bye
b:2:1:bye
c:3:2:1:bye

Of course the patch doesn't break any existent tests.

Please review.

Thanks. Dmitry.

Christian Seiler wrote:
>  Hi,
>  
>  As a followup to the discussion in January, I'd like post a revised 
>  patch to
>  this list that implements closures and anonymous functions in PHP.
>  
>  INTRODUCTION
>  ------------
>  
>  Closures and lambda functions can make programming much easier in 
>  several ways:
>  
>   1. Lambda functions allow the quick definition of throw-away functions
>      that are not used elsewhere. Imaging for example a piece of code that
>      needs to call preg_replace_callback(). Currently, there are three
>      possibilities to acchieve this:
>  
>       a. Define the callback function elsewhere. This distributes code that
>          belongs together throughout the file and decreases readability.
>  
>       b. Define the callback function in-place (but with a name). In that 
>  case
>          one has to use function_exists() to make sure the function is only
>          defined once. Example code:
>  
>           <?php
>              function replace_spaces ($text) {
>                if (!function_exists ('replace_spaces_helper')) {
>                  function replace_spaces_helper ($matches) {
>                    return str_replace ($matches[1], ' ', '&nbsp;').' ';
>                  }
>                }
>                return preg_replace_callback ('/( +) /', 
>  'replace_spaces_helper',
>                                              $text);
>              }
>           ?>
>  
>          Here, the additional if() around the function definition makes the
>          source code difficult to read.
>  
>       c. Use the present create_function() in order to create a function at
>          runtime. This approach has several disadvantages: First of all, 
>  syntax
>          highlighting does not work because a string is passed to the 
>  function.
>          It also compiles the function at run time and not at compile 
>  time so
>          opcode caches can't cache the function.
>  
>   2. Closures provide a very useful tool in order to make lambda 
>  functions even
>      more useful. Just imagine you want to replace 'hello' through 
>  'goodbye' in
>      all elements of an array. PHP provides the array_map() function which
>      accepts a callback. If you don't wan't to hard-code 'hello' and 
>  'goodbye'
>      into your sourcecode, you have only four choices:
>  
>       a. Use create_function(). But then you may only pass literal values
>          (strings, integers, floats) into the function, objects at best as
>          clones (if var_export() allows for it) and resources not at all. 
>  And
>          you have to worry about escaping everything correctly. 
>  Especially when
>          handling user input this can lead to all sorts of security issues.
>  
>       b. Write a function that uses global variables. This is ugly,
>          non-reentrant and bad style.
>  
>       c. Create an entire class, instantiate it and pass the member function
>          as a callback. This is perhaps the cleanest solution for this 
>  problem
>          with current PHP but just think about it: Creating an entire 
>  class for
>          this extremely simple purpose and nothing else seems overkill.
>  
>       d. Don't use array_map() but simply do it manually (foreach). In this
>          simple case it may not be that much of an issue (because one simply
>          wants to iterate over an array) but there are cases where doing
>          something manually that a function with a callback as parameter 
>  does
>          for you is quite tedious.
>  
>      [Yes, I know that str_replace also accepts arrays as a third 
>  parameter so
>      this example may be a bit useless. But imagine you want to do a more
>      complex operation than simple search and replace.]
>  
>  PROPOSED PATCH
>  --------------
>  
>  I now propose a patch that implements compile-time lambda functions and
>  closures for PHP while keeping the patch as simple as possible. The 
>  patch is
>  based on a previous patch on mine which was based on ideas discussed here
>  end of December / start of January.
>  
>  Userland perspective
>  --------------------
>  
>  1. The patch adds the following syntax as a valid expression:
>  
>  function & (parameters) { body }
>  
>  (The & is optional and indicates - just as with normal functions - that the
>  anonymous function returns a reference instead of a value)
>  
>  Example usage:
>  
>  $lambda = function () { echo "Hello World!\n"; };
>  
>  The variable $lambda then contains a callable resource that may be called
>  through different means:
>  
>  $lambda ();
>  call_user_func ($lambda);
>  call_user_func_array ($lambda, array ());
>  
>  This allows for simple lambda functions, for example:
>  
>  function replace_spaces ($text) {
>    $replacement = function ($matches) {
>      return str_replace ($matches[1], ' ', '&nbsp;').' ';
>    };
>    return preg_replace_callback ('/( +) /', $replacement, $text);
>  }
>  
>  2. The patch implements closures by defining an additional keyword 
>  'lexical'
>  that allows an lambda function (and *only* an lambda function) to import
>  a variable from the "parent scope" to the lambda function scope. Example:
>  
>  function replace_in_array ($search, $replacement, $array) {
>    $map = function ($text) {
>      lexical $search, $replacement;
>      if (strpos ($text, $search) > 50) {
>        return str_replace ($search, $replacement, $text);
>      } else {
>        return $text;
>      }
>    };
>    return array_map ($map, array);
>  }
>  
>  The variables $search and $replacement are variables in the scope of the
>  function replace_in_array() and the lexical keyword imports these variables
>  into the scope of the closure. The variables are imported as a reference,
>  so any change in the closure will result in a change in the variable of the
>  function itself.
>  
>  3. If a closure is defined inside an object, the closure has full access
>  to the current object through $this (without the need to use 'lexical' to
>  import it seperately) and all private and protected methods of that class.
>  This also applies to nested closures. Essentially, closures inside 
>  methods are
>  added as public methods to the class that contains the original method.
>  
>  4. Closures may live longer as the methods that declared them. It is 
>  perfectly
>  possible to have something like this:
>  
>  function getAdder($x) {
>    return function ($y) {
>      lexical $x;
>      return $x + $y;
>    };
>  }
>  
>  Zend internal perspective
>  -------------------------
>  
>  The patch basically changes the following in the Zend engine:
>  
>  When the compiler reaches a lambda function, it creates a unique name 
>  for that
>  function ("\0__compiled_lambda_FILENAME_N" where FILENAME is the name of 
>  the
>  file currently processed and N is a per-file counter). The use of the 
>  filename
>  in the function name ensures compability with opcode caches. The lambda
>  function is then immediately added to the function table (either the global
>  function table or that of the current class if declared inside a class 
>  method).
>  Instead of a normal ZEND_DECLARE_FUNCTION opcode the new
>  ZEND_DECLARE_LAMBDA_FUNC is used as an opcode at this point. The op_array
>  of the new function is initialized with is_lambda = 1 and is_closure = 0.
>  
>  When parsing a 'lexical' declaration inside an anonymous function the 
>  parser
>  saves the name of the variable that is to be imported in an array stored
>  as a member of the op_array structure (lexical_names).
>  
>  The opcode handler for ZEND_DECLARE_LAMBDA_FUNC does the following: 
>  First of
>  all it creates a new op_array and copies the entire memory structure of the
>  lambda function into it (the opcodes themselves are not copied since they
>  are only referenced in the op_array structure). Then it sets is_closure = 1
>  on the new op_array, and for each lexical variable name that the compiler
>  added to the original op_array it creates a reference to that variable from
>  the current scope into a HashTable member in the new op_array. It also 
>  saves
>  the current object pointer ($this) as a member of the op_array in order to
>  allow for the closure to access $this. Finally it registers the new 
>  op_array
>  as a resource and returns that resource.
>  
>  The opcode handler of the 'lexical' construct simply fetches the variable
>  from that HashTable and imports it into local scope of the inner function
>  (just like with 'global' only with a different hash table).
>  
>  Some hooks were added that allow the 'lambda function' resource to be 
>  called.
>  Also, there are several checks in place that make sure the lambda function
>  is not called directly, i.e. if someone explicitely tries to use the 
>  internal
>  function name instead of using the resource return value of the 
>  declaration.
>  
>  The patch
>  ---------
>  
>  The patch is available here:
>  <http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-16-1.diff>
>  
>  Please note that I did NOT include the contents of zend_language_scanner.c
>  in the patch since that can easily be regenerated and just takes up 
>  enormous
>  amounts of space.
>  
>  The patch itself applies against the 5.3 branch of PHP.
>  
>  If I understand the discussion regarding PHP6 on this list correctly, some
>  people are currently undergoing the task of removing the unicode_semantics
>  switch and if (UG(unicode)). As soon as this task is finished I will also
>  provide a patch for CVS HEAD (it doesn't make much sense adopting the patch
>  now and then having to change it again completely afterwards).
>  
>  BC BREAKS
>  ---------
>  
>   * Introduction of a new keyword 'lexical'. Since it is very improbable 
>  that
>     someone should use it as a function, method, class or property name, I
>     think this is an acceptable break.
>  
>  Other that that, I can find no BC breaks of my patch.
>  
>  CAVEATS / POSSIBLE WTFS
>  -----------------------
>  
>   * On writing $func = function () { }; there is a semicolon necessary. 
>  If left
>     out it will produce a compile error. Since any attempt to remove that
>     necessity would unecessarily bloat the grammar, I suggest we simply keep
>     it the way it is. Also, Lukas Kahwe Smith pointed out that a single
>     trailing semicolon after a closing brace already exists: do { } while 
>  ();
>  
>   * The fact that 'lexical' creates references may cause certain WTFs:
>  
>       for ($i = 0; $i < 10; $i++) {
>         $arr[$i] = function () { lexical $i; return $i; };
>       }
>  
>     This will not work as expected since $i is a reference and thus all
>     created closures would reference the same variable. In order to get this
>     right one has to do:
>  
>       for ($i = 0; $i < 10; $i++) {
>         $loopIndex = $i;
>         $arr[$i] = function () { lexical $loopIndex; return $loopIndex; };
>         unset ($loopIndex);
>       }
>  
>     This can be a WTF for people that don't expect lexical to create an
>     actual reference, especially since other languages such as JavaScript
>     don't do it. On the other hand, global and static both DO create
>     references so that behaviour is consistent with current PHP.
>  
>     But complex constructions such as this will probably not be used by
>     beginners so maintaining a good documentation should solve this.
>  
>   * The fact that 'lexical' is needed at all may cause WTFs. Other languages
>     such as JavaScript implicitely have the entire scope visible to child
>     functions. But since PHP does the same thing with global variables, I
>     find a keyword like 'lexical' much more consistent than importing the
>     entire scope (and always importing the entire scope costs unnecessary
>     performance).
>  
>  FINAL THOUGHTS
>  --------------
>  
>  My now proposed patch addresses the two main problems of my previous patch:
>  Support for closures in objects (with access to $this) and opcode 
>  caches. My
>  patch applies against PHP_5_3 and does not break any tests. It adds a 
>  valuable
>  new language feature which I'd like to see in PHP.
>  
>  Regards,
>  Christian
>