On Wed, Nov 26, 2003 at 02:23:35PM -0800, Glenn Linderman wrote:
>  On approximately 11/26/2003 2:38 AM, came the following characters from
>  the keyboard of Dave Mitchell:
>  >Closures work at follows: when a sub is created, it captures the currrent
>  >instances of any lexical variables that are referred to inside the sub,
>  >but which are delcared outside that sub. The use of eval often delays this
>  >capturing, so that the required lexicals are no longer available.
>  >
>  >eg this simple example:
>  >
>  >
>  >    {
>  >	my $x = 1;
>  >	sub f1 { print "f1: x=$x\n" }
>  >	sub f2 { eval 'print "f2: x=$x\n"' }
>  >    }
>  >
>  >    f1;
>  >    f2;
>  >
>  >produces the following output:
>  >
>  >f1: x=1
>  >Use of uninitialized value in concatenation (.) or string at (eval 1) line 
>  >1.
>  >f2: x=
>  >
>  >What happens here is that when f1 is compiled, the compiler notices that
>  >the sub f1 makes mention of the outer lexical $x, so f1 gets its own
>  >private reference to that variable. When f2 is compiled, it has no such
>  >mention of $x, so it doesn't also capture $x.
>  
>  A follow-on question here: when f1 gets its own private reference to $x, 
>  I'm assuming that it would still share the value with any other subs 
>  defined in that block that would also reference $x.  That is, that each 
>  such sub would get its own reference, but only one value would exist, 
>  and the subs could communicate through that variable if they chose to. 
>  Experimentally, this seems to be the case.

Each time a block is entered that contains a my declaration, a new instance
of that variable is created (in internals terminology, a new SV is
created). When subs 'capture' a lexical at compile time, they create a
pointer to the current instance. When the block is exited, the block's
reference to the instance is deleted, and if nothing else (such as a
closure) has a reference to it, it is freed. So this instance may indeed
be shared betwen subs, as in 

    sub new_counter {
	my $x = 0;
	return sub {$x}, sub {$x++}, sub {$x--};
    }
    my ($sub_val, $sub_inc, $sub_dec) = new_counter;
 
>  >After the { } block is exited, the interpreter disacards the current (and
>  >only) instance of $x. When f1 is later called, it still has its private
>  >copy of $x, and so can print out its value. When the eval is compiled via
>  >f2, f2 hasn't got a private copy of $x, so the eval tries to grab the
>  >value of the 'real' $x, which is now undef.
>  >
>  >In the current development version of perl, you actually get a warning
>  >when this happens:
>  >
>  >f1: x=1
>  >Variable "$x" is not available at (eval 1) line 1.
>  >Use of uninitialized value in concatenation (.) or string at (eval 1) line 
>  >1.
>  >f2: x=
>  
>  OK, this warning sure would have been helpful to my understanding this 
>  issue... in fact, why is it only a warning, instead of an error... if 
>  the variable really doesn't exist, and use strict is in effect, should 
>  it not be an error?  Although, I'd rather have it work, as you can read 
>  below....

Because it would probably break too much existing code.
 
>  >The moral of this tale is to be careful with evals. They often do funny
>  >things because the compiler can't know in advance what the eval might
>  >contain.
>  
>  Sure, I understand the compiler can't know in advance what the eval 
>  might contain.... but I guess I would have expected, from comments in 
>  the Camel about eval being evaluated in the lexical context where the 
>  eval call appears, that the variables in the surrounding block would be 
>  available.
>  
>  It seems like it would be possible to make it work that way, and that 
>  working that way would produce more useful results.

Consider the following:

    sub X::DESTROY { print "X::DESTROY called\n" }

    {
	my $x = bless {}, 'X';
    }
    print "outside block\n";

This outputs

    X::DESTROY called
    outside block

because $x is destroyed as soon as the block is exited. I hope you'll
agree this is the expected behaviour. Now lets modify it a bit:

    sub X::DESTROY { print "X::DESTROY called\n" }

    {
	my $x = bless {}, 'X';
        sub f2 { eval 'print "x=$x\n"' }
    }
    print "outside block\n";
    f2;

Here f2 and the eval are called after the first and only instance of $x
has already been destroyed. What would you like the eval to do at this
point?
	
    
>  Does anyone think this behavior of Perl/closures/evals is a bug?  One 
>  that is worth fixing?  Or even possible to fix as outlined above?  Or 
>  that will ever get fixed?  Or possibly even documented better, so the 
>  trap can be avoided? 

It's not a bug, but it needs to be documented better. I made a start on
improving the documentation, but life got in the way.

[ By the end of today I've got to write a risk assessment and a get-in
schedule for a production of Macbeth that I foolishly agreed to get
involved in. It's aready 11pm, and I haven't started yet. Instead,
I'm engaged in the displacement activity of replying to emails on p5p ;-) ]

>  The warning in the development version does help 
>  with avoiding the trap, I guess.  I guess the fact that the warning was 
>  added implies that someone besides myself was surprised by the current 
>  behavior....

There were a lot of bugs in the closure code up until 5.8.0. A lot of
these have been fixed in 5.8.1, and even more have been fixed in what will
be 5.10.0. I added the warning as part of fix that correctly handles
the case when the variable should no longer be available (previously
the undef value you got may have been mysteriously shared in odd places).

>  For the moment, I've chosen the workaround of moving all the variables 
>  that I intended to have inside the closure, into global space.  This 
>  caused a few naming collisions, which I was able to resolve fairly 
>  quickly, because all the source code of concern was authored by me :) 
>  This workaround, based on your helpful explanation, is, in fact, a 
>  complete 100% cure for the issue... but it suffers a bit in modularity 
>  vs the intended collection of functions sharing the variables of a 
>  closure.  I suppose an alternative solution would be to wrap a different 
>  package around those definitions, which would give me a 2nd global scope 
>  to contain those variables, which would have avoided the naming 
>  collisions, but also forced me to prefix calls to each of the hundreds 
>  of functions defined in for the package with the package name (or to 
>  export them all).

Another approach is to ensure that the sub that calls the eval
mentions all the lexicals that need to be captured and preserved beyond
their normal lifespan, eg 

    {
	my ($a,$b,$c);
	sub do_eval {
	    { no warnings; $a; $b; $c } # capture lexicals
	    eval $_[0];
	}
    }

I don't know whether this would be practical in your situation though.

Dave.

-- 
"The GPL violates the U.S. Constitution, together with copyright,
antitrust and export control laws"
    -- SCO smoking crack again.