Poor terminology around reference ownership

[markshannon]

Currently we use the terms "borrowed" and "stolen" to describe references, both passed those passed as arguments, and those returned.

Returning a "borrowed" reference is fundamentally different to passing a "borrowed" reference as an argument. Using the same term is confusing.

The term "stolen" is needlessly pejorative, which distracts from the semantics.

[encukou]

Returning a "borrowed" reference is fundamentally different to passing a "borrowed" reference as an argument. Using the same term is confusing.

Could you elaborate?
IMO, the key is to know what you're borrowing from. Borrowing from the caller (for the duration of a C function). But so is borrowing from an immutable object.

[markshannon]

The caller always outlives the callee, so it can lend a reference to the callee.

However, the callee cannot lend anything to the caller as it cannot own anything once it no longer exists.
So the only way a borrowed return value makes sense is if the reference is borrowed from another object passed as an argument.

[encukou]

Yup. For example, if you call PyTuple_GetItem or PyArg_ParseTuple with O, the reference is borrowed from a tuple rather than the callee.

[gvanrossum]

IIUC "borrowed" is only used for return values, e.g. "PyTuple_GetItem returns a borrowed reference to the item" whereas "stolen" is only used for arguments, e.g. "PyList_Append steals a reference to the item passed in".

These concepts, by whatever name, are occasionally useful, when an API is intended to transfer ownership (stealing) or not (borrowing). If we picked a neutral term that applies to both cases, we'd end up stating that most calls that return an object return a new reference, which is then owned by the caller, whereas most calls that take an object as argument don't change the ownership from the caller's POV.

Given the intentional asymmetry in the defaults here I'm not sure that picking a neutral term is actually better. And given that we probably should strive to make all public APIs use the default behavior (i.e., neither borrowing nor stealing) I'm not sure that we need to find replacement terminology.

For internal APIs (e.g. _PyList_AppendTakeRef or _PyLong_GetZero) the concepts will remain useful, but I don't think it's worth trying to change (or even standardize) the terminology used.

[encukou]

And given that we probably should strive to make all public APIs use the default behavior (i.e., neither borrowing nor stealing) I'm not sure that we need to find replacement terminology.

That's a good rule of thumb, but IMO there are many legitimate exceptions.
IMO, we should make sure any non-default behaviour is very visible in call sites, so reviewers see it. That is, we should indicate it in the function's name. And for that, we need good terms.

[markshannon]

We still need a name for the default behavior. "default" doesn't convey any information to the reader.

We also need to decide whether to describe the behavior from the point of view of the caller or the callee.
For API functions it makes sense to use the caller's POV. For callbacks, the callee's POV probably makes more sense.

[gvanrossum]

We will probably be able to keep the term "borrowing", since that's now widely used in other languages (esp. Rust, but I predict other languages will also be talking about "borrow checking"). Agreed that "stealing" is pejorative; "takes a reference" might work?

[hauntsaninja]

"transfers ownership" could be a good way to describe the semantics

[gvanrossum]

So the default behavior would be

• For arguments, ownership is not transferred (caller retains existing ownership)
• For return values, ownership is transferred (caller obtains new owhership)

[encukou]

“Transfers ownership” is good in general!

I'd like to keep using “borrow”, but if you use it you should always explicitly say what is being borrowed from. So

• for the default argument passing behaviour (caller retains existing ownership), the callee borrows from the caller.
• With the current PyArg_ParseTuple(args, "O", &obj), obj is borrowed from the args tuple.
• A hypothetical internal _PyLong_BorrowZero() returns a reference borrowed from the interpreter state.

For transferring ownership of an argument, perhaps give? It's not perfect, but short: _PyList_Append_Give, or even PyTuple_Give(tuple, n, obj). (Though I still kinda like “steal” for its connotations of something nefarious going on.)

[davidhewitt]

Rust uses the term "move" to describe transfer of ownership. This might also be familiar to C++ readers.

[gvanrossum]

Honestly I've always wondered what C++ "move" semantics are for. So I think it's not a great term. It's mostly convenient there because it has the same number of letters as "copy" and contrasts nicely -- buy we don't talk about copying references either in CPython.