Generalized Closure Capture

[mattcalabrese]

Val supports closures with the ability to capture data with let, inout, and sink captures. If I understand correctly, this in effect allows users to safely create an object that can hold a mutable reference to data from the enclosing scope. This is great, however, in practice lots of types need to capture data beyond just closures. Is this considered something that Val also aims to solve? It seems like the same exact technique can be applied to other types in the language just as easily, there just is no syntax for it at the moment.

For instance, coming from a C++ background, something commonly done with tuples is to std::tie together existing objects, which effectively captures elements by reference (with std::tie it happens to be all of them captured by reference, but it is not always the case that all elements are captured in the same way). This operation is used for a variety of reasons, such as to apply existing tuple algorithms to objects that weren't initially declared to be in the same tuple. The most common situation where this occurs is when wanting to perform a lexicographical compare with another tuple (often also one with "tied" elements). An example where mutable captures are useful is with assignment operations, where a user calls std::tie to bring together a series of otherwise unrelated objects and set their values all at once based on the value of some other tuple with similar element types (such as if copying an element of a zipped range). IIUC, all of these cases and others could be elegantly solved in Val if it were to support capture specifiers for tuple elements in a way similar to how they are supported for closures.

Perhaps more complicated to figure out syntax for, but also important for uses in library code, is the ability to have captures within user-defined types. Again, the technique used to track lifetime from the outside would still be the same as for closures, but there would need to be a way to notate the relationship between members and constructor parameters.

Finally, for all of the above situations (including just for closures), it would be useful to denote capture of a function's parameter into that function's returned object. A simple example of this would be for the implementation of a higher-order function foo that takes an existing closure A as a parameter, and returns a new closure of a lesser arity by itself capturing A and passing along a constant as one of A's arguments. This would require some kind of explicit notation on the declaration of foo that describes how A is captured into the returned object.

More use-cases can be provided if necessary.

[kyouko-taiga]

Hi Matt,
I have good news for you. I think Val supports everything you mentioned with remote parts!

TL;DR;

type Foo {
  var bar: remote inout Int
}

public fun main() {
  var s = 10
  var t = Foo(bar: &s)
  &t.var += 1
  print(s) // 11
}

To understand in more details, let's look at the design of Val's lambdas. The type of a lambda is expressed as [E](T1, ..., Tn) -> U, where Ti are its parameters, U its return type, and E its environment.

E is a tuple that denotes the captures of the lambda. For example, the type of f is [{ s: remote let Int }](t: Int) -> Int in the example below.

public fun main() {
  var s = 10
  var f = fun(t: Int) -> Int { s + t }
  print(f(1)) // 11
}

For usability, the exact description of E can be erased or hidden.

• visible [{ s: remote let Int }](t: Int) -> Int
• hidden [some Copyable](t: Int) -> Int
• erased [any Copyable](t: Int) -> Int

So as you can see, the type of the captures is exposed to the type system. Hence, you can define higher-order functions that reason about it. The original purpose of that feature was to avoid dynamic allocation.

The environment of a lambda is nothing fancy, it's just a tuple. That means you can emulate std::tie with remote parts.

public fun main() {
  var s = 10
  var t = "Hello"
  // note: explicit type annotation is required to create remote parts.
  var u: { remote inout _, remote inout _ } = (&s, &t)
  &u.s += u.t.count()
  print(s) // 15
}

You can also use remote parts in any user-defined type, such as a slice. I already showed an example here.

A simple example of this would be for the implementation of a higher-order function foo that takes an existing closure A as a parameter, and returns a new closure of a lesser arity by itself capturing A and passing along a constant as one of A's arguments.

I think we can write this use case as below:

subscript capture<T, R, E>(_ x: T, into f: F): [{x: remote let T, f: remote let F}]() -> R
  where typealias F = [E](T) -> R
{
  fun[let x, let f]() { f(x) }
}

public fun main() {
  let s = 10
  let t = fun(_ a: Int) -> Int { a + s }
  let u = capture(s, into: f)
  print(u()) // 20
}

capture accepts an argument of type T with a lambda of type [E](T) -> R and projects a lambda of type [{x: T, f: F}]() -> R. It has to be a subscript because its result is a projection, not an independent value.

You could create a different version of capture that consumes the argument and the lambda to produce a new independent one.

fun capture<T, R, E: Sinkable>(_ x: sink T, into sink f: F): [{x: T, f: F}]() -> R
  where typealias F = [E](T) -> R
{
  fun[sink let x, sink let f]() { f(x) }
}

[mattcalabrese]

Yes! This looks like it covers the use-cases I care about. Are the only references right now the "language tour" and the specification (spec.md?)? I am okay reading from the specification if that is more complete, although a quick search for "remote" turns up no result. I'd like to start experimenting with Val by porting over some little library bits from C++ and see what kind of shift in approaches I'd need.

[kyouko-taiga]

Unfortunately remote parts (previously called stored projections) have not been fully specified yet. We only have a design document.

I'd like to start experimenting with Val by porting over some little library bits from C++ and see what kind of shift in approaches I'd need.

That would be a truly amazing contribution. The type checker isn't ready to handle important parts of the language, so it might still be difficult to really play with an implementation. But we're working hard on it, stay tuned!

In the meantime, don't hesitate to keep hitting us with use cases.