Useability of new mlkem-native API

[praveksharma]

A lot of KEM usecases (such as TLS), allow preserving state between keygen and decaps; this means that the matrix A can be kept expanded and makes input validation for the secret key unnecessary. This would make KEM operations a lot faster but also raises the question of what the new KEM API should look like. The consensus, when this topic came up during a status call, was that the API should allow OQS user to interoperate as much as possible between the different existing standards.

Here are the proposed new functions for mlkem-native (from PQCP) which shall operate on internal data structures:

crypto_kem_keypair_struct
crypto_kem_keypair_derand_struct
crypto_kem_marshal_pk
crypto_kem_marshal_sk

crypto_kem_parse_pk
crypto_kem_encaps_struct
crypto_kem_encaps_derand_struct

crypto_kem_parse_sk
crypto_kem_decaps_struct

The old API would then be wrappers around these new functions:

crypto_kem_keypair: crypto_kem_keypair_struct + crypto_kem_marshal_pk + crypto_kem_marshal_sk
crypto_kem_encaps: crypto_kem_parse_pk + crypto_kem_encaps_struct
crypto_kem_decaps: crypto_kem_parse_sk + crypto_kem_decaps_struct

Along with these new functions to aid interop:

crypto_kem_sk_from_seed
crypto_kem_pk_from_sk

Does this API miss any particular use case?

[baentsch]

this means that the matrix A can be kept expanded

Just for my understanding: Is this proposal for a new API referring to matrices used in all KEMs or just the one from ML-KEM? So, is this proposal thus effectively calling for the introduction of an algorithm-specific API to OQS or for changing the APIs of all KEMs as per the proposal? If the latter, who is supposed to do all the required code adaptations to those algorithms not coming from PQCP in OQS? If the former, are you effectively calling for a reversal to OQS' promise to supporting all PQC algorithms in the same way?

[dstebila]

I don't think there's a need for us to mirror this API one-to-one. Pravek, Spencer, and I went through this API yesterday in our meeting and our conclusion was that if mlkem-native exposes this API, we would still be able to achieve all the functionality in our currently exposed API, plus deterministic key pair generation à la #2070. If user requests come later for liboqs to expose more of this new granular mlkem-native API, we'll consider those queries as and when they come.

[baentsch]

Thanks for sharing these thoughts. I'm still not clear on what is requested with this issue, then: Introducing for public use a fully OQS-supported new KEM API? Or hiding under the covers of the currently supported OQS-APIs another "proprietary" (PQCP-only) API (along the lines of NVIDIAs proprietary cuPQC API)? Mentioning the latter makes me wonder: How would that cater for/fit with any new KEM API proposal as per this issue? Another adaptation needed? Let's discuss in person as I'm apparently too confused.

[dstebila]

I think this is a request from PQCP for feedback on the new API: will it cause problems, will it be useful? My answers: it won't cause problems for us (we can still do all our current things cleanly as well as things we are planning to do such as deterministic key pair generation); but we don't anticipate using any of the new functionality offered by the new API (except perhaps sk_from_seed) and don't anticipate exposing the _struct APIs in our library.

[baentsch]

ACK. Changed title of issue to align with this. Open question: Is the "OQS consumption" pattern (via copy_from_upstream) still supported by this new API or would OQS have to develop wrappers around it?

[SWilson4]

I don't believe that changes would be necessary for the keypair / encaps / decaps API.

It might be nice to have an additional API function which implements the HPKE DeriveKeyPair behaviour. Note that this is exactly the current keypair_derand API call (i.e., FIPS 203 KeyGen_internal). Otherwise, we would have to string together sk_from_seed / pk_from_sk or maybe keypair_derand_struct / marshal_pk / marshal_sk to achieve this functionality, but IMO this is a very small amount of work to maintain if the API remains stable.