Evaluation API - bulk evaluation

[rs-torsten]

Hi,

I faced with 2 scenarios which is uncovered by specification but they looks very important for dynamic-context paradigm.

Both scenarios actually linked with atomicity of configuration in admin system and data consistency.

1. If we have some configuration in admin we want to have guarantees that multiple calls of flags will be executed on same version of this configuration. Because if we perform two calls to get 'foo' and 'bar' so between requesting 'bar' configuration in admin may be changed and cause inconsistent state due to foo was already received with previous version.

2. One more case related to atomicity but also crucial to avoid multiple request instead single request. For example using some flow we use 2 or more flags. So we need to make 2 or more calls by network to get evaluation for same context (for example targetingKey/userId).
   So in additional to problem related to consistency we also have additional network calls which also can be failed and dramatically increase overall latency to process some business request/payload.

I want propose to add new API method to request flags in bulk mode with some specified evaluation context for all of them.

Looks like:

List/Map evaluations = client.getBulkEvaluation(flagsAndDefaults, evaluationContext)

Where flagsAndDefault should represent name of flag and associated default. It also can be map/dict or some structure.

[thomaspoignant]

Hello @rs-torsten, we talked about that since the beginning of OpenFeature and we have decided not to go in that direction.

The main reason why we don't want to offer an API to do bulk evaluation is because after talking to different vendors in the industry (including LaunchDarkly, Flagsmith and more) we had a feedback that this was not something we should build.
LaunchDarkly even told us that they regret to have built this feature available (now that they have it they continue to maintain it).

For the issue you are talking about, those can be handle directly in the provider itself without requesting a new API on OpenFeature.
For example if you look at the implementation of the OFREP web provider you will see that the flags are retrieved in bulk but we don't expose a bulk API on the different SDKs.

[rs-torsten]

Hello @thomaspoignant, thank you for providing feedback.

Well, I'm actually not found any argument why it should't be built and describe 2 valid scenarios which is actually comes from business.

For example if you look at the implementation of the OFREP web provider you will see that the flags are retrieved in bulk but we don't expose a bulk API on the different SDKs.

Using raw provider instead OpenFeature API broke abstraction layer and looks weird.
But also it can be extension rather than mandatory requirement to implement it - like config changed event.

[lukas-reining]

We talked about that since the beginning of OpenFeature and we have decided not to go in that direction.

Thomas do you remember where this has been? I have problems finding it.

The main reason why we don't want to offer an API to do bulk evaluation is because after talking to different vendors in the industry (including LaunchDarkly, Flagsmith and more) we had a feedback that this was not something we should build.
LaunchDarkly even told us that they regret to have built this feature available (now that they have it they continue to maintain it).

Even for systems that do not implement it, it might be valid to fallback to transparently evaluate each flag at once. It might be relatively slow in that case but if all flags were evaluated anyways, it might be okay to at least have some systems that optimized for that.

I feel like one point against it might be different context expected for each flag, but IMHO the context needed for a flag should in best case always be a subset of the total context.

For the issue you are talking about, those can be handle directly in the provider itself without requesting a new API on OpenFeature.

@thomaspoignant but this would mean that the provider author would need to think of it and also mean that evaluation scoped context can not be used (if the provider does not load rules but the values).

For example if you look at the implementation of the OFREP web provider you will see that the flags are retrieved in bulk but we don't expose a bulk API on the different SDKs.

But there we have the static context paradigm, which does not have evaluation specific context.

To sum up, I feel like this could be beneficial and I do not see contra arguments besides the transparent expense of doing single evaluations below the multi evaluation facade for providers not implementing it.

What do others think? @open-feature/technical-steering-committee @open-feature/maintainers @open-feature/governance-board