Support for Digital Signatures

[gettalong]

I have pushed the first part of the digital signature support to the devel branch.

Reading / Valdiating

The current implementation allows reading and validating digital signatures but not writing them; this will be done next.

The easiest way to test the implementation is by using the hexapdf info command as it has been enhanced to read all digital signatures and output general information about the signature as well as validation information.

As for validating digital signatures: The current implementation only checks a few standard things (e.g. self-signed certificate?) and makes sure that the actual signature is valid. It does not provide a full validation stack as this will differ from use-case to use-case. One can implement the needed validation steps by simply sub-classing an existing signature algorithm handler.

It would help greatly if you could run the hexapdf info command on digitally signed PDFs to see whether HexaPDF's implementation is complete.

Writing

As for writing digital signatures: Since PDF 2.0 deprecated some signature algorithms, HexaPDF will only support the most used algorithm for signing. This should not be a problem in practice.

I'm still unsure how much of the legwork will be implemented in HexaPDF itself and how much one has to do oneself. What I can say is that there will be at least a simple method for signing a document in a standard way where the user just has to provide the certificate+key and the needed information.

If HexaPDF should cover special use-cases, I really would like to hear about them.

Help Needed!

As already said I would greatly appreciate it if you could run the hexapdf info command on digitally signed PDFs and provide input regarding the read/write support - thank you! 🙏

[gettalong]

Ping @earthlingworks

[gettalong]

I have just force-pushed the latest changes to the devel branch which include support for signing a PDF document (multiple times) using a locally available certificate and key. The code should easily be adaptable to sign a PDF using an external service, e.g. using a government API to a sign a PDF using your citizen ID certificate.

Feedback welcome!

[earthlingworks]

@gettalong That's great! Excited about this. I'm out for part of the week but will definitely check it out and give feedback once I get a chance. Thank you!

[earthlingworks]

Also, just noticed that I missed your ping from a couple of weeks somehow, sorry about that. I get a ton of Github messages so probably got buried in between some of those.

[gettalong]

@earthlingworks Okay, I found the problem I had with the local file which most probably also affects you.

If a PDF file containing cross-reference streams and object streams gets a new revision appended and that revision uses traditional cross-reference tables, Adobe and some other proprietary PDF readers will complain that the file is damaged even though it is completely readable and opens fine in all tested free PDF readers.

I will fix this in master and do a new release since this is a major problem.

[gettalong]

@earthlingworks I have released 0.19.2 with the fix and updated the devel branch. In my test case it works now. Could you please try it again?

[earthlingworks]

@gettalong this worked great! What's interesting is that Acrobat does not show any icon or indication that the document is signed, but when I try to change permissions or edit text it says the document is secured and can't be changed.

When I've tried it before with other tools it shows a warning that the document is not secured because it's not using an AATL certificate. In this case, it's not showing a warning or anything but it's also not allowing modification. Super interesting.

And this is what I'm seeing for the document properties: https://share.getcloudapp.com/YEuP5PjB

[gettalong]

• It seems that Acrobat only shows the signature panel once the :append_only signature flag is set on the AcroForm dictionary: doc.acro_form(create: true).signature_flag(:append_only) (before signing). The :signatures_exist flag is automatically set by HexaPDF and I thought that that should do the trick (at least according to the specification). It is probably good practice to also set the :append_only flag since a full save will always invalidate the signature.
  

• What does that document show in the document properties before signing?

• It is possible to restrict permissions when signing, similar to what is possible when encrypting a file. This is not done by default, however.

[earthlingworks]

Ok, good to know, thanks. Document properties before signing are very similar, except there is no mention of the "Changing the Document" property: https://share.getcloudapp.com/12u0pBLQ

[marcomd]

Hi, I can't wait to try this new feature. Could you give me a sample snippet of writing using a certificate?

[gettalong]

Make sure you are using the devel branch of HexaPDF to try this out!

Here is the snippet:

require 'hexapdf'

doc = HexaPDF::Document.open(ARGV[0])
handler = HexaPDF::Document::Signatures::DefaultHandler.new(certificate, key, certificate_chain)
# doc.acro_form(create: true).signature_flag(:append_only) # optional
doc.signatures.add('/tmp/etsi.pdf', handler)

This will open the file given on the command line, instantiate a signature handler with certificate, certificate key and the certificate chain and then write the signed file.

[marcomd]

Thank you Thomas.

Is "certificate" a OpenSSL::X509::Certificate object and "key" the private key?

I don't know "certificate_chain", what is it?

[gettalong]

Yes, certificate is the certificate, key the private key used when generating certificate, probably a OpenSSL::PKey::RSA object. certificate_chain is the chain of certificates leading to the signing certificate, ie. the CA and all intermediate CA certificates. This is optional but when specified checking of the signing certificate can easily be done without the PDF reader needing to download anything

[gettalong]

I'm currently writing the how-to guides and key topic page so that it is easy to get started with digital signatures.

[gettalong]

@earthlingworks @marcomd Do you need the following use-case implemented where hashing the document and then signing it are done in separate steps?

• A PDF file is generated or loaded with HexaPDF.
• The new revision with the dummy signature is written.
• The hash calculation is done after writing the file.
• The hash is sent somewhere else to be signed (e.g. to the web frontend of the user).
• Later the hash is signed and a DER-serialized PKCS7 is returned.
• HexaPDF incorporates the signature at the right place, finishing the signing process.

[marcomd]

Thomas I miss which hash you are referring to.
However, my use case is much simpler and it all ends in one session. In fact, hexapdf is used to visually show (with graphic elements and thanks to your help) that the pdf has been signed and with this new possibility I would like to put a revision that has a "seal" function of the pdf and guarantee to those who open it that has not been tampered.

[marcomd]

This is my use case:

• A PDF file is generated or loaded with HexaPDF.

• The new revision with the dummy signature is written.

• The hash calculation is done after writing the file.

• The hash is sent somewhere else to be signed (e.g. to the web frontend of the user).

• Later the hash is signed and a DER-serialized PKCS7 is returned.

• HexaPDF incorporates the signature at the right place, finishing the signing process.

[gettalong]

Regarding the hash: What is meant by applying a digital signature is encrypting with the private key of the certificate. Since we don't want to encrypt the whole document (as this would needlessly double the size of the resulting, signed document), a hash is calculated from the document and that gets encrypted/signed.

[marcomd]

I tried to use the snipped above but the output pdf does not contain any revision with the signature. I can send you a pdf example with a dummy certificate applied with origami to show you what i expected.

As secondary issues:

1. also wondering if doc.signatures.add ('foo.pdf', handler) replaces doc.write ('foo.pdf') is it correct? If so, is it possible to pass the "optimize: true" parameter?

2. I noticed Canvas#line_with_rounded_corner breaks previous code (based on 0.16) because parameters are changed (which now include "in_radius:".) so i suppose until version 1 only patch versions can guarantee unbreakable changes

[gettalong]

I had a look at the ZIP file you provided. The included certificate_output.pdf already contains the revision with the signature. Opening it in Acrobat Reader or Okular and switching to the signature panel also shows that it is signed. How did you check that it doesn't contain the signature?

If you could provide an example signed with Origami I could have a look at the differences to see if HexaPDF does something wrong.

Regarding your other questions:

1. Yes, all parameters of HexaPDF::Document#write are also supported when signing a file. So you can pass the optimize parameter. Please be aware though that this only makes sense when applying the first signature. When applying any later signature this will most likely blow up the size because the initial revisions are written as is.

2. That method was a private method before and is now a public one with an altered method signature. So no promise was broken. And yes, patch versions should generally be safe whereas new 0.x may include incompatible changes. If there are intentional breaking changes, I generally highlight them in the release notes, e.g. https://hexapdf.gettalong.org/documentation/changelog.html#section-6. Once HexaPDF reaches 1.0 the promise will most likely be similar to the one of Ruby with regards to x.y.z: If x changes, there can be major breaking changes. If y changes, there might be small breaking changes but those are very limited. If z changes, there will be no breaking changes.

[marcomd]

I created a script with which to quickly execute the snippet. The certificate_output.pdf file is already present.

To run it:
certificate.zip

bundle install
bundle exec ruby certificate.rb certificate_input.pdf

[marcomd]

Then I have another question: how to assign the signature to a clickable and visible area on the document?

# Example how i did with origami

text_annotation = Origami::Annotation::AppearanceStream.new
text_annotation.Type = Origami::Name.new("XObject")

signature_annotation = Origami::Annotation::Widget::Signature.new
signature_annotation.Rect = Origami::Rectangle[llx: x, lly: y+height, urx: x+width, ury: y]
signature_annotation.F = Origami::Annotation::Flags::PRINT
signature_annotation.set_normal_appearance(text_annotation)

[gettalong]

Thanks for bringing this up, you can do it like this:

require 'hexapdf'

doc = HexaPDF::Document.open(ARGV[0])
handler = HexaPDF::Document::Signatures::DefaultHandler.new(cert, key, cert_chain) #TODO: use correct values

form = doc.acro_form(create: true) # Make sure a form exists
form.signature_flag(:append_only)

sig_field = form.create_signature_field('my signature') # Create a signature field

widget = sig_field.create_widget(doc.pages[0], Rect: [20, 500, 120, 600]) # Create the widget
widget.flag(:print)
xobject = (widget[:AP] ||= {})[:N] ||= doc.add({Type: :XObject, Subtype: :Form}) # Create the appearance for the widget
xobject[:BBox] = [0, 0, widget[:Rect].width, widget[:Rect].height]
xobject.canvas.font("Helvetica", size: 10).
  text("Certified by signer", at: [10, 10])

sig = doc.add({Type: :Sig}) # set an empty signature and apply it to the field
sig_field.field_value = sig

doc.signatures.add('/tmp/etsi.pdf', handler, signature: sig) # use the created signature

When writing this down it is obvious that this is currently overly complex. So I will change that to become easier and more straight-forward

[marcomd]

Thank you Thomas,

your work is always precious.
I see the signature has been applied after i uncomment this line:

doc.acro_form(create: true).signature_flag(:append_only) # optional

The new code, the more complex one, inserts the box but acrobat reader indicates "Signature field without a signature" and then offers me to sign it. This is interesting and I will consider this possibility but I was wondering how to connect the box to the signature created by code. Consider the scenario where the document has already been signed and I'm just pointing where.

I think here we should refer to the signature above handler(why named handler?) instead of creating a new one, right?

sig_field = form.create_signature_field ('my signature')

[gettalong]

Yes, it seems the signature panel is only automatically activated if the :append_only flag is also set. However, the signature is still there even if this flag is not set. You just need to manually open the signature panel in Acrobat.

Hmm... when testing this locally I see the following

Then when I click on the signature appearance the following dialog pops up

So there actually should be a valid signature there and inspecting the file confirms that.

Do you have an example file that works as you expect in Adobe Reader? If so, could you provide that for inspection?

It is named "handler" because it should handle the signing process. The default implementation is very simple but custom implementations could do more complex things.

When digitally signing a PDF, you actually need two things for this to work: The signature dictionary which will contain (or contains) the cryptographic signature. And the signature field which is used for the visual display of the appearance and/or to facilitate signing in a PDF reader.

In my sample code sig_field is the signature field and sig is the cryptographic signature object that will hold the signature once signed.

[marcomd]

Question 1. link the signature field to the signature created by code

Now it is more clear to me, thanks, but I don't understand how to link sign to handler

sig = doc.add({Type: :Sig}) # set an empty signature and apply it to the field

Question 2. using a valid certificate

This is not a hexapdf issue but it's generic and related to self-signed certificate.
I have read this adobe documentation which explains how to create it.
After creating it you will get a PKCS12 file with the .p12 extension and then:

pkcs12 = OpenSSL::PKCS12.new(File.read("your_identity.p12"), "your_password!")
private_key = OpenSSL::PKey::RSA.new(pkcs12.key.to_pem)
certificate = OpenSSL::X509::Certificate.new(pkcs12.certificate.to_pem)

Using that certificate your signature will be valid

[gettalong]

1. The handler is currently independent of the signature dictionary, so there is no link. However, I plan to extend the functionality so that the handler can modify the signature and signature fields before the actual signing, e.g. to automatically provide a custom appearance if none was set.

2. I think what Adobe does is create a standard self-signed certificate and import that into its certificate store. Therefore it will appear as valid for you but not for anybody else. You can achieve the same effect by creating the self-signed certificate in any other way and then in the signature dialog of the signed PDF you get the chance to trust the certificate.

[marcomd]

1. Ok
2. Ouch, you are right! To obtain a certificate valid for all others, it is necessary to purchase it. Here the list of trusted members

[marcomd]

Sorry but my acrobat is in italian and i can't change language.

The pdf created with hexapdf include the signature (with the method above it is valid) and the field for a new sign. Acrobat shows "Signed, all signatures are valid. Fill in the form below ..."

Then, if a click on the signature field acrobat let me choose an identity for a new signature but what i would is to show the signature details. I hope I explained myself.

[gettalong]

Hmm... this is different from the result I get when viewing in Adobe Reader, e.g. mine shows me exactly the dialog with the signature details, like you want

However, the native Adobe Reader on Linux is Adobe Reader 9 which is very old. I will try to install a newer of Adobe Reader via Wine and see what it says there.

If you open the signature panel, do you see the signature there?

[marcomd]

No, the section showing the details of the signatures is empty. In fact it is strange because the signature should be in the list even if "invisible".

[gettalong]

Okay, I tried a few things and when not adding a signature appearance, the signature panel works again. So there is something about the appearance that Acrobat doesn't like. Acrobat also throws error 23 when trying to execute the "validate all signature" command if we include the appearance. Still trying to find out why.

[gettalong]

@marcomd I found a solution to this: If the optional /P entry in the signature field is set, then the signature panel works correctly. Still hoping this will work in all cases but at least in the ones I tried it produced the correct result in Adobe Acrobat.

[gettalong]

@earthlingworks @marcomd I have pushed some more changes to the devel branch.

Basic code now:

doc = HexaPDF::Document.open(ARGV[0])
sig_field = doc.acro_form(create: true).create_signature_field('my signature')
widget = sig_field.create_widget(doc.pages[0], Rect: [20, 500, 120, 600])
widget.create_appearance.canvas.
  stroke_color("red").rectangle(1, 1, 99, 99).stroke.
  font("Helvetica", size: 10).
  text("Certified by signer", at: [10, 10])

handler = doc.signatures.handler(reason: 'Reason', certificate: CERTIFICATES.signer_certificate,
                                 key: CERTIFICATES.signer_key,
                                 certificate_chain: [CERTIFICATES.ca_certificate])
doc.signatures.add('/tmp/signed.pdf', handler, signature: sig_field)

• There is the additional doc.signatures.handler method for getting a signing handler. And it now allows to set reason, location and contact information.
• To specifiy doc.write options use the write_options argument instead of specifying them directly in the doc.signatures.add call.
• Creating an appearance can now be done using widget.create_appearance.

[gettalong]

@earthlingworks @marcomd I have just released version 0.20.0 with digital signature support. See

• https://hexapdf.gettalong.org/news/2021/hexapdf-0-20-0.html
• https://hexapdf.gettalong.org/documentation/changelog.html and
• https://hexapdf.gettalong.org/documentation/howtos/signing-pdfs.html