[Proposal]: Extension interface implementation

[agocke]

Discussion for proposal #9319

[agocke]

Responding to one of the comments from @HaloFour

These rules sound very similar to the coherence rules enforced by Rust. My opinion is that such limitations are fatally limited in that I expect that the vast, vast, vast majority of use cases expected by users of a feature like this would be very precisely to extend a type nor declared in the current project with an interface also not declared in the current project.

Rust's rules are called orphan rules and they are very similar to the rules that I listed in the linked discussion. However, the motivation is not the same, and it turns out this is closer to "coincident evolution" than a mirroring of ideas.

Rust has orphan rules primarily because they want to guarantee particular notions of consistency in their type system and avoid some challenges with typing in a system where multiple implementations can "cross" each other.

In contrast, the orphan rule I propose here is mainly about implementation choices. Since the very beginning of .NET there have been two things that have been built heavily into the runtime: interface implementation is part of type identity, and type identity is fixed. Changing either of these things is almost impossible in .NET. You start messing with type identity and things blow up. You start trying to separate interface implementation from type identity, things blow up.

What the orphan rule brings to .NET is the ability to adjust the interface implementation list without changing type identity. Because everything is coherent and entirely in the view of the runtime at type load time, the runtime can effectively shim any set of interface implementations into existing types without changing identity. That's not to say this is an easy task, but it's at least understandable. This would still be an incredibly tricky change to the runtime, but I think it's possible.

Absent an orphan rule, I'm skeptical that any alternative is going to progress beyond brainstorming. For anyone who is interested in tackling that challenge: you need to start at the ECMA-335 level. Your task will be to walk every single rule regarding type identity, conversions, interface implementation, interface dispatch, generics, and reflection, and describe precisely how they will behave in your design. At minimum, I expect you will need to answer the following problem: List<T> where T doesn't implement some interface, List<T> where T does implement some interface, and List<T> where T does implement some interface, but with a different implementation, must all coexist at runtime. T will nominally appear to be the same type. List<T> will also need to sometimes be the same type and sometimes not. You cannot modify List<T> itself in any way, as the code is already written. How will your system handle all of those circumstances?

[HaloFour]

To bring my two comments over for context: 😄

These rules sound very similar to the coherence rules enforced by Rust. My opinion is that such limitations are fatally limited in that I expect that the vast, vast, vast majority of use cases expected by users of a feature like this would be very precisely to extend a type nor declared in the current project with an interface also not declared in the current project.

My other issue with a design like this is that by impacting the type at runtime you lose all of the scoping mechanisms that exist for extensions in general. IMO, "extension implementation" should have similar scoping rules, in that I can influence them based on the namespaces/classes imported in that compilation unit. Obviously that requires the compiler to be involved in the design as the runtime would have no concept of that.

I've had a number of conversations very recently about what I think are the major strategies for implementing something like "extension implementation" which is why this is very front of mind. 😄

Absent an orphan rule, I'm skeptical that any alternative is going to progress beyond brainstorming.

That depends on what the goals are. If the goal is to affect the runtime so that the target type always appears to implement said interface, sure, these rules are necessary. But I don't think that is the only approach.

Take Scala implicit classes, for example. They allow you to bridge any arbitrary type to any arbitrary trait through a witness class that is the implicit class. This is done entirely by the compiler, and has scoping behavior that is practically identical to extension methods. As long as the implicit class is in scope, you can effectively treat the target type as if it implements that interface. To me, this is the behavior I would expect from something that aligned itself with "extensions".

I think a role-based approach would work very differently, too, especially if a "role" was a kind of type that the runtime understood and supported. Being very hand-wavey (both because I haven't thought about it a huge amount and because I don't know nearly enough about the inner workings of the runtime) but I would envision a special type that appeared to extend the receiver type (even though it might not be possible due to the receiver type being sealed, a struct, etc.), that would also extend from the various interfaces. That "role" type would be reflected in the metadata, but it would end up elided away by the runtime and virtual calls to the interface would be patched up to the extension implementation declared as a part of that role.

Implementation details aside, I think it's much more important to focus on the use cases that users actually want to solve. Any use case I've seen here involved a BCL type and a BCL interface. I was talking to someone on the Discord channel just a couple of days ago who wanted the ability to make arbitrary BCL types appear to implement IEnumerable<T> or IEquatable<T>. None of those use cases can be solved by an implementation that requires coherence an orphan rule to ensure safety.

[HaloFour]

To be very clear, I am not arguing against your proposal. I think it makes a lot of sense and I agree that the orphan rule is probably necessary. I would expect that it would be the only real practical way to attach interfaces to an external type from a runtime perspective. 😄

[agocke]

Take Scala implicit classes, for example

Yeah I think this is a good alternative solution. Unfortunately I’m also a bit skeptical that the kind of reflection-heavy code that .NET users write would play well in that world. But it’s an alternate design point (although worth mentioning that it’s controversial even within scala).

Implementation details aside, I think it's much more important to focus on the use cases that users actually want to solve

I don’t really buy this argument. If there isn’t an implementation strategy that can easily solve all possible requirements, the implementation has to play a role in the discussion. Fundamentally we will always be constrained by what’s practical. This is engineering, after all.

[HaloFour]

Yeah I think this is a good alternative solution. Unfortunately I’m also a bit skeptical that the kind of reflection-heavy code that .NET users write would play well in that world.

Totally agree. Scala's approach works really well when you want to quickly treat a type like a trait, but it doesn't help with reflection or even basic type checking. It's definitely more of an implicit one-way conversion between the receiver type and the target interface. I have no idea if any kind of runtime support could help here.

I don’t really buy this argument. If there isn’t an implementation strategy that can easily solve all possible requirements, the implementation has to play a role in the discussion. Fundamentally we will always be constrained by what’s practical. This is engineering, after all.

Yeah, that makes sense. At least go down the rabbit holes enough to understand the pros and cons of each approach and which use cases they can potentially solve. I really don't think there is an obvious "winner" here, and it's possible that two (or more) approaches could be adopted side by side.

[fabianoliver]

I like this proposal.

Massively subjective of course: Personally, I don't seem to recall any case where I would've needed or liked to implement some third party interface for some third party type. If I ever had to, I'd imagine that effectively wrapping objects into some decorating type, either statically or via dynamic proxy and mixing in some new interface implementation would typically get the job done?

Similarly, I'm not a huge fan of the idea of scope-bound extension interface implementation just yet. Makes code harder to follow (granted, same can be said about extension methods). And too many question marks for my liking.. say you have a function taking a parameter of type object. Would a "obj is ISomeExtensionInterface" check succeed if the caller of said function had the extension in scope? If the extension is in scope for the function itself? What if the type check was reflection based?
There are undoubtedly clear answers to all of that, but it doesn't seem like something that's easy to intuit, which IMHO is a big negative for any feature (Or maybe I'm missing something fundamental and those are non-issues?!)

Anyways, back to this proposal here: Conditional interface implementation would be quite nice to have. Especially for functional code - think of option or result types that could implement iequatable, icomparable etc if and only if their inner type does as well.
Is that worth a complex change to the runtime? I can't judge that, but I could've definitely used that feature on occasion. And as of now, I unfortunately don't think there's much of a way to fudge your way around it's absence easily.

For understanding, could the orphan rule be relaxed further? For example, could you allow an extension for an external interface of some external type if it's defined in the application's entry assembly - I'd imagine maybe that might be scannable before implementing type or interface is loaded?

[agocke]

I've updated my proposal to give an example of a practical use case that could be implemented even with the orphan rule, but can't be implemented today.

[ds5678]

@HaloFour I have a slightly cursed workaround.

interface IDisposable2
{
    void Dispose();
}
readonly ref struct Disposable<T> : IDisposable
    where T : IDisposable2, allows ref struct
{
    private readonly T _t;

    public Disposable(T t) => _t = t;

    public void Dispose() => _t.Dispose();
}
static class Extensions
{
    extension<T>(T t) : IDisposable2
        where T : IDisposable
    {
        void IDisposable2.Dispose() => t.Dispose();
    }
    extension<T>(T t)
        where T : IDisposable2
    {
        public Disposable<T> AsDisposable() => new Disposable<T>(t);
    }
    extension(int i) : IDisposable2
    {
        void IDisposable2.Dispose() { }
    }
}
class Program
{
    static void Main()
    {
        using (1.AsDisposable())
        {
        }
    }
}

A lot of the boilerplate could be source generated.

[HaloFour]

@ds5678

That's basically the wrapper type approach, emitting a witness that implements the target interface. But at that point the IDisposable isn't actually an int, it's a Disposable<int>, and reflection would not indicate that int implements IDisposable. In my opinion this approach would probably work for many/most use cases, although you wouldn't need quite so much chicanery.

@agocke

As for this proposal, I do really wonder how it would work with generics. Could you have an extension that implements an interface across T? Or what about specialized generics, like IEnumerable<int> but not IEnumerable<T>?

[agocke]

I think the precise rules are up for discussion as long as we preserve three properties: uniqueness, visibility, and stability.

Uniqueness: there must be exactly one best implementation. In the case that ambiguous implementations are visible at type load time, an exception must be produced by the runtime.

Visibility: the runtime must be able to determine the correct implementation only by loading the assembly containing the interface and the assembly containing the target type.

Stability: additional loads may not change the results.

So the rule against overlapping is primarily about uniqueness. If a rule exists preferring one implementation over another, that doesn’t seem like it inherently violates the rules.

[agocke]

Also,

Could you have an extension that implements an interface across T

As long as the interface and extension are in the same assembly, that seems fine.

[jkoritzinsky]

I know the proposal explicitly doesn't cover IDynamicInterfaceCastable, but I highly recommend using it as prior art when it comes to actual implementation. For example, emitting the implementation of the extension as an interface instead of a class is not only much safer (no GC or debugger confusion from the type of this differing from the method table of this), it also would follow the well-trodden path that IDynamicInterfaceCastable has proven works for years.

Another interesting option related to IDynamicInterfaceCastable would be to extend the ExtInterfaceImpl table to have a "Conditional" column. If the value in the "Conditional" column is 1 then the cast would only succeed if IDynamicInterfaceCastable returns true for IsInterfaceImplemented. Additionally, the runtime could provide (either as a DIM or as a helper method) a special implementation of GetInterfaceImplementation that looks up the implementation in the ExtInterfaceImpl table. This would allow the runtime to enforce the invariant of "a given type may have only one implementation per object type, but any given object can say whether or not it implements the interface" through usage of metadata.