Help with PKCS7

[koehn]

I have a server attached to an HSM that I want to use to generate PKCS7 for a PDF. Basically the client generates an SHA-256 digest and the server creates the SignedAttributes, signs them with its key, and returns the resulting PKCS7 for the client to insert into the PDF.

But I'm running into a problem where the PDF doesn't verify and I cannot understand why. Here's the code I'm using to create what I think should be a valid signature:

    public static byte[] signPdfDigest3(Certificate[] certificateChain, PrivateKey privateKey, byte[] pdfDigestBytes,
            String provider) throws Exception {
        PdfPKCS7 pdfpkcs7 = new PdfPKCS7(privateKey, certificateChain, null, "SHA256", provider, false);
        return pdfpkcs7.getEncodedPKCS7(pdfDigestBytes, Calendar.getInstance(), null, null);
    }

Basically, I load the certificate from the HSM and get a PrivateKey that represents the private key (which remains locked away in the HSM). The pdfDigestBytes are the sha256 digest calculated by the client (I have manually verified these by splicing together the PDF minus the placeholder with two dd invocations and sending it to sha256sum and the digest matches). The resulting DER is exactly what I would expect. I can see a call to the HSM, although it's for Sha256WithRsaSignature which isn't what I was expecting, since the resulting DER says it contains RSA.

I spliced apart the DER, pulled out the signedAttributes, signature, and certificate, and tried to verify the signature using openssl, and sure enough it doesn't verify.

If I use a test harness to create a PKCS7 and then immediately verify it using keys generated with BouncyCastle, the verification fails.

Am I using the API incorrectly? Is this even achievable?

[andreasrosdal]

Is this even achievable?

Yes, I think so.

Does this code work for you?

public static byte[] signPdfDigestProperly(Certificate[] certificateChain,
                                           PrivateKey privateKey,
                                           byte[] digest, // digest from client
                                           String provider) throws Exception {
    String digestAlgorithm = "SHA256";

    // Construct PdfPKCS7, but don't provide the content
    PdfPKCS7 pdfpkcs7 = new PdfPKCS7(privateKey, certificateChain, null,
                                     digestAlgorithm, provider, false);

    // Generate the authenticated attribute bytes (DER-encoded SignedAttributes)
    byte[] authenticatedAttributes = pdfpkcs7.getAuthenticatedAttributeBytes(digest, null, null, CryptoStandard.CMS);

    // Use HSM to sign the authenticated attributes (this is what gets signed in a PKCS#7 detached signature)
    Signature sig = Signature.getInstance("SHA256withRSA", provider);
    sig.initSign(privateKey);
    sig.update(authenticatedAttributes);
    byte[] signedBytes = sig.sign(); // actual digital signature from HSM

    // Set the external digest manually
    pdfpkcs7.setExternalDigest(signedBytes, null, "RSA");

    // Now encode the PKCS7 structure (including SignedAttributes, Signature, Certificates)
    return pdfpkcs7.getEncodedPKCS7(digest, null, null, null, CryptoStandard.CMS);
}