Explicit backing fields

[merfemor]

This is a discussion of the "Explicit backing fields" feature. The text of the proposal can be found here.

The proposed feature addresses the request KT-14663 "Public and private type for the same property".
With proposed changes, this can be achieved by declaring property field type explicitly, which allows to automatically provide smart casts to field types in calls within the private scope.

---

Summary of changes for those familiar with previous versions of the proposal:

• The new proposal focuses mainly on exposing read-only supertypes use case (e.g. MutableLiveData + LiveData, MutableStateFlow + StateFlow), while rejecting support for more comprehensive ones (like MutableStateFlow + asStateFlow() wrapper)
• This results in a significant reduction in design complexity and resolves concerns raised in the previous proposal, namely the "two backing fields" use case (e.g., when asStateFlow() wrapper is cached instead of calculating on each call), serialization challenges, poor composability and unclear overload resolution rules

[MartinHaeusler]

It's good to see that this issue has not been forgotten and I think it is very important.

I would like to re-iterate that this topic isn't limited to getters (even though the solution described in the KEEP seems to focus on them); it's about setters (and the types they accept) as well.

It is one of the most annoying and frustrating limitations in Kotlin right now that I cannot express this:

private var user: String
    set(user: String) { field = user }
    set(user: User) { field = user.username }

There are countless examples in DSLs and otherwise where this would be extremely helpful. Since we're decoupling the backing field type from the "public" type, I would propose to take this into consideration as well. In Java, this has always been possible via regular setter overloading and it was a frequently used pattern. Kotlin properties lack this type of flexibility at the moment.

There's this issue on YouTrack which was created 11 (!) years ago. The issue links to the former discussion (#278) and the discussion links again back to the YouTrack issue. It's going in circles.

Please, Jetbrains 🙏 It's nothing fancy, but this is really needed.

[zarechenskiy]

I don't think it's related to "explicit backing fields", at least in the current design. So I’d kindly ask to keep this discussion focused on "explicit backing fields" for now.

However, it's a good feature — thanks for bringing it up. Setter overloading isn't in our short-term plans, but it could be a nice and reasonable extension of properties.

[PavloBohdanSonerim]

Having skimmed through the new proposal, it left me a little confused.
Correct me if I'm wrong, but will this be the first-ever field type in Koltin that won't support inheritance?
What about OOP?

Illustration to the issue:

open class BaseViewModel : ViewModel() {
    val loadingState: StateFlow<Boolean>
        field = MutableStateFlow(false)
    
    fun commonRequest() {
        loadingState = true
        ...
        loadingState = false
    }
}

class FeedViewModel : BaseViewModel {
    fun loadFeed() {
        loadingState = true // NO ACCESS
    }
}

Suggestion:

open class BaseViewModel : ViewModel() {
    val loadingState: StateFlow<Boolean>
        protected field = MutableStateFlow(false)
    
    fun commonRequest() {
        loadingState = true
        ...
        loadingState = false
    }
}

class FeedViewModel : BaseViewModel {
    fun loadFeed() {
        loadingState = true // Has access
    }
}

[zhelenskiy]

But what stops you from declaring the explicit backing field protected?

[PavloBohdanSonerim]

@zhelenskiy
https://github.com/Kotlin/KEEP/blob/explicit-backing-fields/proposals/explicit-backing-fields.md#visibility

private is the default and the only allowed visibility for explicit backing fields.

[IRus]

I would say in proposed example you don't need to access loading state anyway, instead do something like this:

open class BaseViewModel : ViewModel() {
    val loadingState: StateFlow<Boolean>
        field = MutableStateFlow(false)

    fun commonRequest() {
        loadIntoModel {
            // Your code here
        }
    }

    protected fun loadIntoModel(block: () -> Unit) {
        loadingState = true
        block()
        loadingState = false
    }
}

class FeedViewModel : BaseViewModel {
    fun loadFeed() {
        loadIntoModel {
            // load feed
        }
    }
}

[PavloBohdanSonerim]

@IRus this WILL work but in a way, this is a hack

[PavloBohdanSonerim]

@IRus and this is a very simplified use case. Flexibility is still required, as there are a lot of cases where you have to handle the loading state

[fvasco]

I I understand well, defining field uninitialized and initialize it in the init block is not currenlty supported.

class SomeViewModel : ViewModel() {
    val city: LiveData<String>
        field: MutableLiveData<String>

    init {
        city = MutableLiveData()
    }

Is there a reason (or oversight) for this exception?

I suppose that, for simple cases, it is still possible to write: field = run { MutableLiveData() }

[zarechenskiy]

To me, it seems to be an oversight. @merfemor WDYT?

[merfemor]

I see no problems allowing that. I'll mention it explicitly in the KEEP.

[rnett]

Not supporting non-private fields is quite disappointing to me, and I don't see a particular rational why not to include it. There's a number of use-cases already listed in the doc, and the only reason not to support it given is

The existing workaround with backing property pattern is intuitive and relatively concise, and therefore it sets a high bar of convenience and simplicity for the new feature's syntax. That's why, when you're trying to support as many use cases as possible but not overcome "conciseness bar", you often end up with something too implicit and "magical."

I don't think the ability to write

public val city: LiveData<String>
    protected field = MutableLiveData()

is particularly "implicit" or "magical", especially when you can already do the same for getters and setters. In fact, supporting protected set for mutable properties but not (mutable) backing fields seems more confusing and unexpected. This could even be supported by having protected set on a property with a backing field apply to the field (IMO this would be quite confusing and I don't actually suggest it, but it's an option).

[zhelenskiy]

As far as I read the doc, it states that it is possible by writing the visibility modifier, but the default field visibility is private.

[merfemor]

The main obstacle here is that the syntax is misleading: when you see "protected field" in the property declaration, it makes you think that the backing field visibility in compiled code is protected. While in reality protected is the visibility of automatic smart cast, and backing fields are always private.

[PavloBohdanSonerim]

@merfemor how will this impact the developer's experience?

[merfemor]

@PavloBohdanSonerim it leads to wrong expectations for the person reading the code.

The thing here is that it is a problem of this particular syntax with field keyword, because it ties smart-casting to backing fields, which are meant to be private-only. This could be potentially overcome by an alternative syntax, but so far no elegant solution has been found (see section "Naming discussion" in KEEP).

[CLOVIS-AI]

Could the proposal clarify its stance on:

val city: StateFlow<String>
    field = MutableStateFlow("")
    get() = field.asStateFlow()

The proposal seems to imply this isn't supported (there is no example with both a field and a get(), a similar example is in the 'not supported' case) but the grammar does seem to imply it is supported.

As a user, I would find very surprising if this wasn't supported (it means this new feature is incompatible with the existing features). If it is supported, I don't understand why the asStateFlow() example is listed as unsupported.

About the use-case itself: at least in the codebases I've worked on, when I use the backing property pattern, it is much more common that the aim is to expose a read-only wrapper than just upcast the value. If this use-case is not supported, most backing property usages will not change, which decreases the usefulness of this feature a lot.

[zarechenskiy]

It's explicitly unsupported. You can think of this feature as "computed-once" properties, which allow the compiler to apply additional smart casts within the scope where the backing field is accessible. As a result, custom getters are not allowed — including in this case.

As for this use case more generally: the most well-known example is List/MutableList, where we return the same object and not a wrapper. And to my knowledge, this hasn’t caused any issues or user reports. The StateFlow/MutableStateFlow case is more debatable. In our documentation, we sometimes recommend using .asStateFlow(), but in practice, for example, in Android, there's no such guideline: https://developer.android.com/kotlin/flow/stateflow-and-sharedflow

In my view, the core problem is that there's no reliable way to enforce the use of .asStateFlow() without changing the design or adding checkers. And originally, .asStateFlow() was introduced to support property declarations with omitted expected types — not primarily to improve safety.

Back to the feature itself: we've gone through several rounds of discussion around this case and ultimately decided to drop this case. It introduces too much hassle for too little gain — requiring two backing fields, unclear smart cast applicability, potential need for explicit syntax to access underlying fields, and so on. In the end, syntax alone doesn't fix the pattern — it's still easy to forget to use .asStateFlow() or its equivalent. At this stage, if you need more backing fields, then two good old properties are the way to go.

However, we still might address this case by improved support for delegates. See my response here: https://youtrack.jetbrains.com/issue/KT-14663/Support-having-a-public-and-a-private-type-for-the-same-property#focus=Comments-27-11853379.0-0 Ideally, it should be possible to encapsulate the logic about asStateFlow completely, so people don't have to think every time that they need to return a wrapper.

[CLOVIS-AI]

It introduces too much hassle for too little gain — requiring two backing fields, unclear smart cast applicability, potential need for explicit syntax to access underlying fields, and so on.

Sorry, I'm not following. Why would there be two backing fields?

As for this use case more generally: the most well-known example is List/MutableList, where we return the same object and not a wrapper. And to my knowledge, this hasn’t caused any issues or user reports.

Returning a wrapper with .toList() is somewhat common in shared Kotlin/Java codebases, because up-casting to List doesn't do anything from the Java side, so it's not sufficient to avoid mutability. See also KT-66257.

[serras]

Why would there be two backing fields?

In your example not, because you're using a getter, but in most cases what people want is actually to compute the (non-mutable) StateFlow once.

val city: StateFlow<String> = field.asStateFlow()
    field = MutableStateFlow("")

// simulates

private val _city: MutableStateFlow<String> = MutableStateFlow("")
val city: StateFlow<String> = field.asStateFlow()

[...] up-casting to List doesn't do anything from the Java side [...]

There are two interrelated questions here.

The first is whether those wrappers are the actual solution to people actually subverting the type system. If you look for example to Android docs, they do not recommend using .asStateFlow() at all, simply using a different type annotation.

The second is whether explicit backing fields should be the way to streamline such wrapping. Because allowing any potential expression to be part of the field -> getter conversion imposes important constraint on the design. In particular, since we're no longer sure that the field and the getter refer to the same underlying value, there's a bunch of analysis we can no longer perform successfully (think of anything related to smart casting or exhaustiveness, for example).

[pdvrieze]

@serras I would say that the way that these complex cases (with multiple fields) would be best through optimization of delegates:

• Allow access to the delegate (e.g. city::delegate)
• Allow optimizing the object layout to inline it into the container (this may already be done automatically by the jvm)

[pdvrieze]

From a design perspective I'm not sure that the right approach is to see this as an application of smart casts (even though most of the underlying reasoning applies). While this may simplify implementation in the short term it may make the longer term evolution (to some of the other desired features) difficult. Maybe a consideration of the restrictions would be good instead:

• Non-final property: This creates both implementation and (developer) reasoning complexities. Requiring the property to be final is probably correct (overriding properties with backing fields can be an issue anyway)
• Specialised return type in subclass (specialisation without code change) without a generic parameter. This is the most common case for non-final field-backed properties (a generic type would work, but could lead to large amount of generic types) - perhaps the best solution here is to have a generic implementation base class and a public interface without the generics.
• Setter: The problem becomes whether you directly access the field or the setter. Semantically using a setter should be fine though, except that the field type and setter parameter may mismatch. Setter overloading would help/fix this, especially if it would allow the case where the read type has no setter, but the field type does.
• Getter: Technically would require symbol resolution to resolve differently from in the scope than externally. The problem is that you make one symbol effectively become two dependent on where it is used from - this can very quickly become confusing for users.
• Different field type: If the field is not visible outside the getter/setter it should be valid to have a different field type as long as there are getters/setters. While overloaded setters could allow more flexibility on setters, this would not expose the field type to the containing class.

Fundamentally the issue is that in properties the field is not only private, it is hidden except in the getter/setter - separately its type must be the same as the interface (getter). This could be addressed by making the field addressable in either of three ways:

1. Having a separate property as backing field
2. Having syntax to access it: e.g. myProperty::field
3. Allowing for the case were we expose the field as a supertype in conjunction with restrictions that make this non-confusing (even if rules could make it non-ambiguous that would not imply non-confusing per se) – this is the current proposal

While the currently proposed restrictions for case 3 could perhaps be loosened a bit they will always be "restrictive". This means that either solution 1 or 2 must be used. Solution 1 will always be available. Solution 2 would need to be added to the language. It has two drawbacks:

• It adds to the language load and would quickly become a cornercase feature (not known/used by a large amount of users)
• The actual textual representation is longer than the current convention: _myProperty vs myProperty::field
  There are also some advantages:
• There is a clear semantic link between the field and the property
• It could be used to expose the backing fields for delegates (for example to check whether a lazy property has been set before using it in a toString)
• It allows for explicitly avoiding the use of getters/setters in the implementation (even in cases where the backing field is not explicit or has a different type) – but contrary to the separate property it makes this semantic very clear (clunky by design).

[zarechenskiy]

Nice analysis, thanks!

One of the things we’ve observed is that introducing syntax for accessing the underlying field tends to work worse than just two plain properties. With a special syntax, we also need to be cautious about how it interacts with overrides, smart casts, and the scopes where such a field is accessible.

Fundamentally the issue is that in properties the field is not only private, it is hidden except in the getter/setter - separately its type must be the same as the interface (getter)

To offer another perspective: we also explored connecting this feature to the idea of immutability. From the compiler’s point of view, it might look like we’re simply exposing the backing field. But in practice, the goal is to return immutable or read-only views, like MutableList -> List, LiveData -> MutableLiveData, SharedFlow -> MutableSharedFlow. The property’s return type is just one such bound, but a similar intention could be applied more broadly, for example, to functions that return immutable views, or parameters that require defensive copying. However, in practice, we seem to need a simpler solution as we don't always need defensive copying, so the part with immutability we want to address slightly differently: KT-77734 Better immutability in Kotlin

In the end, we decided to focus on things that were most problematic in previous approaches, which is smart-casts. And while right now the feature is called “explicit backing fields”, it’s really about enabling more precise smart casts — there’s nothing fundamentally new about backing fields themselves. A more accurate but strange (?) name might be something like:

// Always automatically smart-cast this property to MutableList in the private scope
val users: List<Name> = mutableListOf() 
    private smart-cast

// Always automatically smart-cast this property to MutableList in the protected scope
// However, the backing field is still private!
val users: List<Name> = mutableListOf() 
    protected smart-cast 

This is similar to what we have with variables, but with a broader scope:

fun test() {
    val x: Any
    x = "string"
    
    x.length // smartcast to String
}

[lunakoly]

Unfortunately, the following example from "§ Call from inline functions" won't work:

class C {
  val prop: List<Int> field = mutableListOf()

  inline fun foo() {
    prop.add(1) // ERROR

    if (prop is MutableList) {
        prop.add(2) // OK
    }
  }
}

It will, but that's a current bug in the compiler (see KT-7972): prop is MutableList must only yield prop: MutableList<out Int>, which shouldn't allow putting anything inside the collection.

[merfemor]

Thanks for noticing! I've edited that part.