byname.md

Pointers

This is a proposal for adding pointer types to Python, analogous to pointers in "C".

A pointer is a regular Python object with certain new operators defined.

types.PointerType is added to the types module. It is an abstract class.

collections.abc.TargetPointer[T] is a pointer which "points to" a target of type T. collections.abc.MutableTargetPointer[T] also supports assigning and deleting the target.

Creation

A pointer is created by the expression &(target). target is any expression. The pointer is a sort of live proxy for the target expression.

The creator of the pointer is the scope containing the &(target) expression.

Name Binding

Any simple names in target, or recursively in any tuple or list elements, are bound as local variables in the creator, unless declared global or nonlocal.
The compiler treats these bound names as local variables, even if there is no other binding operation in that scope.
Any other names in target will be treated normally. They could be implicitly or explicitly global, free, or (if bound elsewhere in the scope) local.

Dereferencing

Explicit

A pointer is dereferenced by the expression *(pointer).

• As a simple expression, \*(pointer) evaluates to the current value of the target in the creator. This can be different from the value of the target at the time the pointer was created. Any exception generated by evaluation of the target is propagated to the original expression.

• The statement \*(pointer) = value executes the statement target = value in the creator, and any exception is propagated to the original statement. A SyntaxError is converted to a NotImplementedError if target = value is not valid assignment statement. target can have at most one starred item at any level of nesting.

• The statement \*(pointer) += value executes the statement target += value in the creator, and likewise for all other augmented assignment statements. It is equivalent to:

  • temp = *(pointer)
  • temp += value
  • *(pointer) = temp

• The statement del pointer executes the statement del target in the creator, and any exception is propagated to the original statement. This includes a NotImplementedError if del target is not valid del statement. target can have no starred items at any level of nesting.

Implicit

A function or lambda parameter can be implicitly dereferenced by declaring it is &param. All occurrences of param will be treated as *param. This can be done at multiple levels.

Syntax Additions

• &(target)
  Creates a pointer that represents target.
  Parentheses are not required if target is a primary.

• *(pointer)
  Refers to the same target which created pointer. This is a unary operator. Parentheses are not required if pointer is a primary. Otherwise, pointer is an expression which evaluates to a pointer object.

• &parameter
  In a function def or a lambda expression, signifies that parameter is a pointer to a target. This is equivalent to &parameter in "C++".
  The & may be repeated, meaning that the parameter is a pointer to another pointer, which may be another pointer, etc.
  With a varargs parameter, the * precedes the &(s).
  All references to parameter are dererenced implicitly, once for each & in the specification.
  Note, the equivalent in "C" is *parameter rather than &parameter. We use &parameter in Python because *parameter already has another meaning.

Grammar Additions

• factor:
  | '&' factor
  | '*' factor

• param:
  | '&' param

Special Method Names

The PointerType class has these special method names. You can make your own pointer class by defining these methods.

• object.__getderef__(self). The value of *object.
• object.__setderef__(self, value). Sets new value of *object.
• object.__delderef__(self). Deletes value of *object.

Any class with __getderef__ will be a subclass of TargetPointer.

Any class with all three will be a subclass of MutableTargetPointer.

Are Pointers Useful?

There are already various ways in which a function can be called which will communicate a result to a caller. Examples:

• Pass a parameter which is a list, and then store the result in list[0], or pass an index into the list as another parameter.
• Pass a dict, and store the result in an agreed upon key, or pass the key as another parameter.
• Pass some object, and store the result in an agreed upon attribute, or pass the attribute's name as another parameter.
• Create the function with a def or lambda and store the result in a local or free variable of the caller. The function needs to have the name of the variable, and this won't work for a local variable in a class statement, or a call to exec() with an explicit locals dict.
• For several results, return a tuple, or a namedtuple, or make the function into a generator.

So it could be argued that this Pointer mechanism is not needed.

However, these mechanisms require some degree of knowledge in the called function of the mechanism desired by the caller.

What we'd like is to have a single name in the called function represent the result to be sent back to the caller, without knowing the actual mechanism.

Passing a pointer to the function is the simple way of doing this. The function can dereference the pointer either implicitly or explicitly, and the caller decides what the actual target of the pointer is.

Pointers provide power and flexibility beyond the usual mechanisms:

• The target of a pointer can be an expression, whose value may change over time. Derefencing the pointer follows changes in the value of the expression. Also, with an attribute or subscript target, the target follows changes to the identity of primary object whose attribute or subscript is referred to.

• The target of a pointer has a longer lifetime than that of the scope in which it was created.

• The target of a pointer can be, or use, a local variable in a class definition.