Make our Converter Interface Converter<S,T>

[radcortez]

Right now, our Converter interface is generic type for the conversion result, but the conversion original value is fixed to a String.

It might be interesting for our top level conversion to support a generic type on the original value as well, like:

public interface Converter<S, D> extends Serializable {
    D convert(S value);
}

The motivations behind this is to better support requirements in MP JWT. Original issue here:
microprofile/microprofile-jwt-auth#100

The source object from where values are retrieved from is a Json structure (the JWT). Values are represented as the Json counterparts (JsonObject, JsonNumber, etc), so in fact, our conversion needs to call .toString on these and perform additional operations back and forward for the final conversion to take place.

With String conversion

Object claim = jsonWebToken.getClaim(claimName);
String value = claim.toString(); // this does Integer.toString(num);
Integer integer = converters.convertValue(value, Integer.class); // and we convert from a string to int again.

With Source and Target

Object claim = jsonWebToken.getClaim(claimName);
Integer integer = converters.convertValue(value, Integer.class);

JsonNumberToIntegerConverter implements Converter<JsonNumber,Integer> {
  Integer convert(JsonNumber value) {
   return value.intValue();
  }
}

[radcortez]

@dmlloyd What do you think of this?

[dmlloyd]

Are you expecting that converters get automatically chained? If I've registered a Converter<Foo,Bar> and a Converter<Bar,Baz>, do I automatically have a Converter<Foo,Baz>? Why or why not?

[radcortez]

I'm not so sure if we should go that route yet. I fear that if we go that route, we may become another full object mapper library like MapStruct, ModelMapper, Dozer, etc.

We may reach the conclusion that we will need chaining, but I think not at this stage. I think that it is acceptable that if you require conversion of composed objects, it should be handled manually calling the appropriate converts in the entry point converter.

What do you think?

[dmlloyd]

I'm not sure I have an opinion yet, just questions. :-)

Another case that I wonder about is collection conversion. How would it work? Today since converters are one-dimensional, we can easily define every possible conversion in terms of other converters (for example we can compose optional, list, and a scalar type all into one chain) in a very straightforward way. But what would be the correct composition approach here? Would we have a converter from a String to a List<String> and then a converter from List<String> to List<T>? How would other parametric types work?

One feature that SmallRye Config misses that I think would be powerful would be automatic composition. For example I should be able to register a global converter like this:

public final class MySetConverter<T> implements Converter<Set<T>> {
    private final Converter<T> delegate;
    public MySetConverter(Converter<T> delegate) {
        this.delegate = delegate;
    }
    public Set<T> convert(String input) {
        // do the conversion here using my custom Set impl
    }
}

Then the config framework examines the signature, notes I have a type var T and that the constructor has a corresponding parameter of type Converter (or Class), and registers the set implementation as a default for all sets, eliminating the need for special compositional factory methods like we have today (see smallrye/smallrye-config#40).

Is such a feature even possible when converters are two-dimensional? I'm not so sure.

[radcortez]

Most likely Collections conversion will need to be handle differently. We need to only require the element type converter of the Collection to be implemented, and then we handle the rest.

This means that for these cases we only support Collection<S> to Collection<D> conversion. Not sure how useful would it be to support S to Collection<D> automatically, unless you supply the specific converter of course.

Then you can have specialised Converter types, for instance String like we have today, and keep the same behaviour.

I see no reason for smallrye/smallrye-config#40 wouldn't work for two-dimensional types. In theory, you should able to implement that. On practice, it may provide too much work :)

[dmlloyd]

I don't mean two dimensional arrays, I mean two dimensions of typing.

The canonical theoretical problem is like this: if I have a Converter<S, Foo> and a Converter<Bar, D>, and I want to convert a Bar to a Foo, which converter is used, and why?

[radcortez]

Ops. Sorry. I meant types not arrays. Fixed.

That is an interesting question. I would probably lean to a Converter<S, Foo> for the sole reason that your result should be the most accurate one. Of course, you cannot guarantee that, but I think the Converter focus should always be in the the type being returned, because that is the one consumers are probably more interested in, or why would they need to convert it in the first place? :)

[dmlloyd]

Ops. Sorry. I meant types not arrays. Fixed.

That is an interesting question. I would probably lean to a Converter<S, Foo> for the sole reason that your result should be the most accurate one. Of course, you cannot guarantee that, but I think the Converter focus should always be in the the type being returned, because that is the one consumers are probably more interested in, or why would they need to convert it in the first place? :)

Sure, but that's a subjective argument, and thus the exact opposite argument could be made with equal validity. The fact is that both converters have equal validity from any objective measure, therefore whatever option is chosen will be wrong to some extent. And who wants to use wrong software?

But we don't need to give up all hope; every problem is solvable. But before we go into that, do we have other use cases beyond Config and JWT that need to be evaluated?

[radcortez]

Sure. But we are here to fix the issues we took the wrong decision. I don't loose hope with you here :)

Right now, I've looked into this with the JWT perspective, because it was an obvious use for the library and even have open issues about supporting Converts like Config.

Maybe OpenTracing / OpenTelemetry could also benefit from these, so you could inject and convert to types directly from the trace context (but we will be dealing with String here).

[dmlloyd]

OK so we have two cases; in one case, the input is a String and in the other the input is some structured type of known complexity.

I would go a little farther and say that considering smallrye/smallrye-config#216 and other YAML-related issues, plus microprofile/microprofile-config#550, we might consider that configuration sources also would want to use structured types of known complexity rather than dealing just in raw strings all the time.

So for the sake of exploring this idea, let's assume that is the case, and let's assume we have some intermediate representation or representations that encapsulate all of the possible structured types.

Now things are interesting because we can have an InputConverter<T> which converts a T to the intermediate type(s), and an OutputConverter<T> which converts the intermediate type(s) to T.

From the perspective of config, configuration sources could produce the intermediate type(s) instead of String, and that solves the aforementioned issues quite cleanly. From the perspective of JWT, a converter from JSON to one or more of the intermediate type(s) would be available which could potentially solve that case as well.

The final feature of such a design is that the converter ambiguity problem is solved definitively. Every combination of input and output can be supported equally well because every converter type is one-dimensional.

But, what would the intermediate type(s) look like, and why? That's the next challenging question.

[dmlloyd]

It's also worth nothing that InputConverter<T> is not necessary.

[radcortez]

And do you have any ideas of the intermediate types? Quite frankly, I couldn't think on any acceptable intermediate type.

Unless I didn't understand the concept fully, in practice you will have a 2 step conversion. One from source to the intermediary type and another one from the intermediary to destiny. Is that correct?

[dmlloyd]

And do you have any ideas of the intermediate types? Quite frankly, I couldn't think on any acceptable intermediate type.

Unless I didn't understand the concept fully, in practice you will have a 2 step conversion. One from source to the intermediary type and another one from the intermediary to destiny. Is that correct?

Right, if you do both halves of the concept - unless your intermediate type already includes your conversion source (or target). For example if the intermediate type (or one of the intermediate types) is a string, then you don't need an input converter to convert from string.

[radcortez]

Ok, let go for a more complex case with the JsonObject. Lets say I want to convert a JsonObject into an Address.

So I have an InputConverter<JsonObject> and a OutputConvert<Address>, right? In this particular case it seems redundant to have an intermediary type between JsonObject and Address, since JsonObject is already a ready parsed representation to convert to something else.

Do we have specialised OutputConverters like JsonOutputConverter<Address> that uses JsonObject as the intermediary type? (maybe the name is misleading).

[dmlloyd]

On the one hand, it could be possible to have a structured intermediate type that can (like JSON or YAML for example) be an integer, FP number, string, boolean, or a list or map of any intermediate type. This solves the problem elegantly, in a way: the input could be JSON, or YAML, or a plain string; the value becomes the intermediate structured type; the output converter interprets the open ended type and produces an Address instance.

However, some context is lost. Perhaps interpretation of an Address differs between JSON and YAML due to different specifications. Perhaps there's more than one interpretation of Address even within JSON due to different standards specifying different formats.

So, this surrounding context would have to be carried forward somehow. The input converter would have to know where the information comes from in some standard way that can be interpreted by the output converter in a general way. This mechanism would have to be built to prevent the output converter from generating an object wrongly due to an unrecognized input context.