Hi Frédéric,

On 12 May 2008, at 23:48, Frédéric Bonnet wrote:
>  [...]
>  But first, I'd like to point out a misunderstanding. The discussion  
>  was
>  not around argument passing conventions, but around references in data
>  structures. So all the proposals around proc enhancements are a bit
>  off-topic, however interesting they might be. (Actually I planned to
>  make the same kind of proposal in Cloverfield). More on that below.

OK, I didn't get that from your original post.
[...]
> > Yes. "Everything is a string" is fundamentally incompatible with the
> > idea of strong references, as far as I can tell (i.e. it is
> > impossible to create a properly abstracted system of reference using
> > strings as references). Tcl_Obj internal rep hacks are not just prone
> > to failure, but also break EIAS.
> 
>  True. However Cloverfield tries to address this issue by providing  
>  a way
>  to serialize references more robustly. See the Tridekalogue for more
>  information:
> 
>  	http://wiki.tcl.tk/20643 (see [11.7])

How is it any more robust than a variable name? Surely you have  
exactly the same problems.

> > [...]
> > What then is the difference between these weak references and just
> > names? Are they looked up in a different context/namespace?
> 
>  There is no difference, only syntactic. Cloverfield introduces a whole
>  set of access methods for variable values, e.g. vector or map (list or
>  dict in Tcl lingo). However the parser can get in the way of the  
>  access
>  syntax. For example:
> 
>     # Same as puts [lindex $a 1 2 3]:
>     puts $a{1 2 3}
> 
>     # Fails because the parse splits a{1 2 3} at spaces and build 3  
>  words.
>     set a{1 2 3} 4
> 
>     # The above must be written using the weak reference syntax:
>     set @a{1 2 3} 4
> 
>  The original discussion revolved around whether to extend the  
>  parser so
>  that words must span across braces as a general rule (i.e. a{1 2 3} is
>  one word and not 3), or whether to require special syntax whenever
>  variables must be indexed.

This seems a separate discussion from the present one. Why not insist  
that variable names be single words (they almost always are), and  
then use ${a 1 2 3} or [set {a 1 2 3} 4]?

>  [..]
> 
>  However Tcl has no means of storing references outside of plain
>  variables. For example, one cannot build a structure that holds
>  references other than using variable names (which must be global in  
>  most
>  cases). This causes a number of problems in term of lifetime  
>  management
>  that OO system writers know too well. Cloverfield tries to address  
>  this
>  limitation.

I don't really understand what benefits these references have over  
just using unique globally-scoped variable names. Perhaps some  
examples would help. If I put a reference into a dict (map) and then  
send that over a socket, what gets serialised at the other end --  
does the reference get recreated, or just the current value at time  
of serialisation? i.e.

set d [dict create foo 12 bar $&jim]
puts $sock $d
... on other end ...
set d [read $sock]
set $&d(bar) ... ;# is that the right syntax?

Does that work? I assume for that to work that the string rep of a  
reference must identify itself as a reference (e.g. the "{ref foo} 
bar" syntax), but then how do I get just the current value from a  
reference if $myref returns the reference?

If the calling conventions stuff is secondary, and the primary aim is  
to get garbage collected first-class references that can be stored in  
data structures, then surely Jim has already solved this problem?  
(http://wiki.tcl.tk/13847)

> 
> > 2.) Generalise the notion of what can be referred to by a Var to that
> > of a general "resource" -- i.e., not just Tcl_Objs, but also "opaque"
> > entities like commands, channels, objects, etc. This would allow for
> > finer control over the lifetime/scope of these resources (e.g. proc-
> > local commands or channels), and could be extended to encompass
> > things like general reference-counting of resources and perhaps even
> > a generalised serialisation framework (i.e. $foo means "attempt to
> > serialise the resource referred to by the var named foo").
> 
>  That's an interesting proposal. However it raises deep philosophical
>  questions about lifetime management of said resources. Other paradigms
>  are sometimes better suited (C++ RAII for example).

I don't think it does raise any deep philosophical issues, only  
practical ones. What I am suggesting is to allow the scope of various  
opaque things (objects, commands, aliases, channels etc) be allowed  
to be confined to smaller scopes than just global/namespace.  
Essentially, I am proposing something not too distant from RAII. For  
instance, you could have a proc-local XOTcl object:

proc foo {} {
    myclass create bar ...
    bar mymethod ...
} ;# bar is destroyed on proc exit (RAII)
proc foo varName {
    upvar 1 $varName bar
    myclass create bar ...
    bar method ...
} ;# bar is preserved in reference passed

Note that this means unifying command/var scopes (and others) -- at  
least, that's the nicest way of doing it. Of course, as you point  
out, this says nothing about storing resources/references inside data  
structures, but I don't think the solutions to this problem do any  
better than a global variable name.

Incidentally, are references in Cloverfield just first-class  
variables? I.e. do they refer to just Tcl_Objs, or can they also  
refer to the various opaque resources I mention above (e.g. XOTcl  
objects, channels etc)?

>  ==============================
>  [...]
>  With references, the caller can choose whether to pass by value (in a
>  purely functional way) or by reference:
> 
>     % set v1 {a b c}
>     % set v2 [linsert $v1 1 d]
>     a d b c
>     % set v1
>     a b c
>     % linsert $&v1 1 d
>     a d b c
>     % set v1
>     a d b c
> 
>  This eliminates the value/reference dichotomy that exists in current
>  Tcl. But as I said above, this is just a side effect. The primary  
>  intent
>  is to build complex structure patterns (with potential circularities).

As I pointed out before, eliminating this distinction at the level of  
implementation would need more than just a bit of extra syntax: each  
command would have to be written to be able to do sensible things  
when passed either a value or a reference. Could you show some  
example code of how a Cloverfield command that works in this way  
would be implemented in C (or point me towards the source if there is  
an implementation available)? I think that would help clarify what is  
intended.

This dichotomy itself isn't unique to Tcl. It is a simple use/mention  
distinction that is fundamental in most languages. You can either  
have different syntax for whether a variable is being used (i.e. the  
value retrieved) or mentioned: e.g. $foo vs foo or foo vs 'foo  
(Lisp), or you can make the distinction at a higher level in the  
syntax, e.g. foo = foo + 1 (C), where foo before the '=' (so-called  
"lvalue") is taken as a reference, and after is taken as a value, or  
you can use the "mention" syntax everywhere and then provide a  
function to retrieve the value: foo vs [set foo] (original Tcl). A  
final option (ML) is to provide only the "use" syntax (and therefore,  
only constants) and then provide a separate reference object type  
that has it's own use/mention distinction: !foo vs foo. Your proposal  
still has this distinction: $foo vs $&foo, so you haven't eliminated  
the dichotomy, but duplicated it! Rather than clarifying things, I  
think this will just make it much more confusing.

I personally *like* the fact that there is a clear distinction  
between commands that are purely functional (guaranteed not to  
produce side-effects, no matter what they are passed) and commands  
that are obviously effectful.

[...]

>  Neil Madden wrote:
> > How can $&foo syntax guarantee that a variable named foo exists any
> > more than upvar can?
> 
>  Twylite got it right:
> 
>  Twylite wrote:
> > Upvar can't.  Upvar links to the variable irrespective of whether it
> > exists or not.  If you try to dereference the local name and the  
> > given
> > variable doesn't exist in the caller's scope then you get an error.
>  [...]
> > Errors are handled in the right context, and the error is
> > thrown for the right context and with the right name.  And your  
> > proc can
> > treat the variable as a (local) parameter, while the caller gets to
> > determine if it should be passed by value or as a reference (in which
> > case changes made by the proc are available to the caller).
> >
> > Upvar means that errors are discovered in the proc when you  
> > dereference
> > the linked variable.  The programming error was when you passed in  
> > the
> > wrong variable name.  The error message you get (with upvar) is
> > misleading because it indicated the linked variable name in the proc,
> > not the name of the linked-to variable (which is in a different  
> > scope).

Right, so it doesn't guarantee anything at all. It just catches the  
error earlier and produces a nicer error message (which I showed can  
be done in Tcl now).

>  [...]
> 
>  Neil Madden wrote:
> >> set b alpha -> alpha
> >> set c [lappend b beta] -> alpha beta
> >> puts $b -> alpha beta
> >> puts $c -> alpha beta
> >>
> >> What if I need "alpha" and "alpha beta", i.e. the original $b and
> >> the final $c?
> >
> > Then use [linsert] instead of [lappend]:
> 
>  That's exactly the kind of things newcomers find confusing as hell in
>  Tcl, the fact that we need two sets of commands to carry on the same
>  operations. Granted, most of them come from all-reference  
>  languages, and
>    are understandably puzzled by Tcl's copy-on-write and pass-by-value,
>  but they rightly feel that Tcl's way of faking references with  
>  variable
>  names is bizarre at best.

They are not the same operations! Most other languages don't have  
this distinction as most operations are pass-by-reference and you  
have to do extra work to get a side-effect free version. So, in Java  
you'd write:

     myList.append(myElement)

in Tcl:

     lappend myList $myElement

What Java typically has no equivalent of is:

     linsert $myList end $myElement

and, indeed, you'd need a separate method to get this, even in the  
append case. Other languages provide only the side-effect free  
versions, and you have to go to extra lengths to mutate a variable  
(e.g. myList := !myList @ [myElement], in ML). So yes, people are  
confused to find a language that happily supports both ways of doing  
things, but I don't see how introducing yet another way could  
possibly make this any simpler.

Ideally, I would like the ML situation: names map to things  
(resources) and never change, and then you introduce a mutable  
reference type as a type of resource. For a language like Tcl I don't  
think this can work well on the level of values without doing serious  
abuse to EIAS, which is why I'd prefer to do it at the level of named  
things. This makes it hard to use with value data structures (lists/ 
dicts) etc, so the idea would be that those remain as purely- 
functional structures, and instead when you need a complex mutable  
structure you build it in an OO system, making use of better  
facilities for scoping such resources.

[...]
>  Neil Madden wrote:
> >> No - you're still missing one thing: with implicit upvar the proc
> >> controls the semantics (pass-by-value or pass-by-reference); with $&
> >> syntax the caller is in control.
> >
> > I didn't miss that. I strongly think that the proc should control the
> > semantics, as the proc has to implement them.
> 
>  In some cases, procs should be in control. In other cases the caller
>  should be. As of today Tcl provides no support for the second class of
>  problems.

What is wrong with the [foo -var bar] vs [foo $bar] option I  
described before?

> > The proc must always be written to know whether it is getting a value
> > or a variable reference. Otherwise allowing callers to pass in a var
> > will result in implementation-specific undefined side-effects being
> > performed on the variable.
> 
>  Unless it's done consistently and properly documented. The current
>  situation is no more consistent on this point, with its mixture of
>  pass-by-value and pass-by-name conventions.

It is! Everything is pass-by-value (pass-by-name is something else  
entirely). Some commands expect values directly for some arguments  
and variable names for other arguments. They never expect either for  
the same argument.

[...]

-- Neil
This message has been checked for viruses but the contents of an attachment
may still contain software viruses, which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Tcl-Core mailing list
Tcl-Core@[...].net
https://lists.sourceforge.net/lists/listinfo/tcl-core