Please add support for asynchronous order finalisation

[dharasis1993]

This asynchronous flow has always been specified in RFC8555
https://datatracker.ietf.org/doc/html/rfc8555#:~:text=A%20request%20to%20finalize%20an%20order%20will%20result%20in%20error,to%20the%20%22certificate%22%20field%20of%20the%20order.%20%20Download%20the%0A%20%20%20%20%20%20certificate.

[hslatman]

@dharasis1993 could you elaborate on what exactly should be changed? It's not entirely clear whereto your link points, nor is it clear what's missing in our implementation.

[dharasis1993]

I wanted to know whether Small Step supports Async order finalisation or not. As per the code -

certificates/acme/order.go

Lines 310 to 321 in df13aae

	cert := &Certificate{
	AccountID: o.AccountID,
	OrderID: o.ID,
	Leaf: certChain[0],
	Intermediates: certChain[1:],
	}
	if err := db.CreateCertificate(ctx, cert); err != nil {
	return WrapErrorISE(err, "error creating certificate for order %s", o.ID)
	}
	o.CertificateID = cert.ID
	o.Status = StatusValid

Here the cert is created instantly in /finalize api and status of order is changed to valid. What if a client time-out during this phase? Is there any way that the client can poll the order?

[hslatman]

@dharasis1993 you're right that the certificate is signed "synchronously" there as soon as the client requests the /finalize endpoint. But in the full ACME flow this is still considered asynchronous, because it'll only happen when the client sends the POST request, which can be some time after the challenge was solved. Polling the Order happens before that.

[dharasis1993]

What if the db.CreateCertificate takes time to return? In that case the client may be timed-out. I have seen these in case of Apple acme client.

[hslatman]

Gotcha, so you're looking for support of this part of the RFC:

   o  "processing": The certificate is being issued.  Send a POST-as-GET
      request after the time given in the Retry-After header field of
      the response, if any.

db.CreateCertificate could indeed take a bit of time, as could the signing process. I haven't seen timeouts at this stage of the process with various ACME clients myself, including the Apple ACME client, but I may be using a system that does everything more than quick enough to it never result in a timeout in this part of the flow. Note that we're already using this in our platform to issue certificates to Apple clients.

If issuance is indeed really slow, I would try to look into finding the root cause of slowness and see if that can be improved. Also I would be curious if the Apple ACME client does support the processing state correctly. Without knowing that we don't know whether or not rewriting this part of the ACME issuance process is going to provide much value. Determining if it does could be done by hacking our code a bit by returning processing instead of valid, and running some code after a set time to turn the Order into valid after a while. That won't be acceptable for the actual implementation, but at least it should give you insight into the Apple ACME client behavior.

[dharasis1993]

As of now Apple client don't support processing state. Though i am searching a formal documentation from Apple on this regards. But definitely it can be enhanced by Apple. The hacking seems little difficult as far as nonce and signature verification are in place. Finalising order async'ly may enhance the overall certificate issuance success rate.

[Xkynar]

In our use-case, we noticed cert-manager has a hard time-out on the /finalize request after ~1 minute waiting, but if you return the order as Processing it will respect the "Retry-After" header and poll for the Valid order.
For us this is essential since the PKI behind our acme server sometimes take longer to issue a certificate (some CAs seem to be quite slow, for example GlobalSign integrations seem wobbly and may require a couple retries).
This would be a necessary feature for those cases.

[dharasis1993]

Agree. It is a valid use-case.

[Xkynar]

Hey @hslatman,
Any updates on the status of this issue? Is it being considered?
Thanks!

[hslatman]

Hey @Xkynar,

I would say it's still under consideration, but it's not a trivial change to get this to work across our stack, and so far we haven't seen the need for this functionality grow beyond the people active in this thread. It's good to know at least one ACME client that supports the flow like this, though, which should help in verifying a future implementation.

[Xkynar]

@hslatman Sure, cert-manager (or anything that uses golang's crypto acme package) is a good example:

• In here we see cert-manager /finalize implementation, which calls golang.org/x/crypto acme CreateOrderCert
• CreateOrderCert calls WaitOrder if the order status is not "valid" (i.e. processing)
• WaitOrder performs postAsGet requests to whatever comes in the finalize's Location Header (should be the /order endpoint) until it's no longer "processing".
  • We can see in the spec (section above link) that the /finalize answer should include this Location header.
• WaitOrder respects the Retry-After header otherwise it queries every second.
• After the Order is finally in a final state, it will proceed with the certificate download.

So in practice any client using "CreateOrderCert" will hold for as long as the server needs to process the Order asynchronously.
Does this help?

[hslatman]

So in practice any client using "CreateOrderCert" will hold for as long as the server needs to process the Order asynchronously. Does this help?

Absolutely; thank you 🙂

On our side, it's still something non-trivial to implement, as it involves managing some additional state, which will have to be plumbed through in a few places in our stack. But with the above, it at least becomes easier to test.