Safe Evaluation Mode

[salvalcantara]

Problem Statement

Given potentially incomplete data, I want CEL expressions to evaluate negatively instead of failing.

Examples

For the following JSON payload:

{
  "a": [ 
    {"b": 1, "c": [1, 2, 3], "d": "foo"},
    {"b": 2, "d": "bar"},
    {"c": [0]}
  ]
}

I want these expressions to evaluate to false, for example:

size(aaa) > 0
aaa.exists(x, x.b == 1)
a.exists(x, x.b == 3)

As of now, all will fail due to missing fields. An obvious solution is to make the expressions safe beforehand like this:

has(aaa) && size(aaa) > 0
has(aaa) && aaa.exists(x, has(x.b) && x.b == 1)
has(a) && a.exists(x, has(x.b) && x.b == 3)

But that's precisely what I want to avoid.

Proposed Solution: Safe Evaluation Mode

The main idea would be to replace unresolvable attributes with null.

For a.b == 1 that would give us null == 1 which would evaluate to false as desired.

The problem is when you have something more like this a.b > 1 because that would be transformed into null > 1 and then evaluation would fail with a no such overload error due to the null type not supporting the > operator. The same might happen for other operators/functions. To account for this, the interpreter could consider an evaluation type of middleware which simply inspects if there are any (non-)errors like those due to missing fields and if that's the case simply return false.

Rough Sketch

Optionally, decorate the interpreter like this:

public class SafeEvalLib implements Library {
  @Override
  public List<EnvOption> getCompileOptions() { return Collections.emptyList(); }

  @Override
  public List<ProgramOption> getProgramOptions() {
    return List.of(ProgramOption.customDecorator(
        interpretable -> {
          if (interpretable instanceof SafeEvalAttr) {
            return interpretable;
          }
          if (interpretable instanceof Interpretable.EvalAttr) {
            var evalAttr = (Interpretable.EvalAttr) interpretable;
            return new SafeEvalAttr(evalAttr.adapter(), evalAttr.attr());
          }
          return new SafeEval(interpretable);
        }
    ));
  }
}

where:

• SafeEvalAttr would do the replacement with null for nonresolvable attributes
• SafeEval would act as a middleware to ignore the (non-)errors caused by SafeEvalAttr (and potentially others)

The former would be a trivial tweak of EvalAttr, while the latter would look like this:

public class SafeEval implements Interpretable {

  private final Interpretable delegate;

  public SafeEval(Interpretable delegate) {
    this.delegate = delegate;
  }

  @Override
  public Val eval(Activation ctx) {
    Val result = delegate.eval(ctx);

    //
    // Safe evaluation: falsify some (non-)errors
    //
    if (Err.isError(result)) {
      var val = (String) result.value();
      if (val.contains("no such overload") || val.contains("undeclared reference") || val.contains("expected iterable type")) {
        return Types.boolOf(false);
      }
    }

    return result;

  }

  @Override
  public long id() {
    return delegate.id();
  }
}

This safe evaluation mode would be disabled by default, but could be optionally enabled by the user.

Relates to:

• https://github.com/google/cel-spec/wiki/proposal-246

[salvalcantara]

Instead of replacing unresolvable attributes with null, one could use a singleton Undefined (or Missing) data type along the lines of Optional Chaining in JavaScript. By overloading all the operators for this data type to simply propagate the undefinedness (missingness), the SafeEval interpretable above would not be necessary, bettering the overall design from a type perspective. Alternatively we could support Optionals (proposal 246):

• https://github.com/google/cel-java/blob/main/common/src/main/java/dev/cel/common/types/OptionalType.java
• https://github.com/google/cel-java/blob/ece46c4e7dd69a86b1bc3ec9416fdfdfc4761a48/extensions/src/main/java/dev/cel/extensions/CelOptionalLibrary.java