Decoding nested ASN.1 sequences with implicit tags

[onyn]

Hello. Got into an interesting situation while playing around with ASN.1. I use org.bouncycastle:bcutil-jdk18on:1.80.

Here's my code:

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import org.bouncycastle.asn1.ASN1Encodable;
import org.bouncycastle.asn1.ASN1InputStream;
import org.bouncycastle.asn1.DERSequence;
import org.bouncycastle.asn1.DERTaggedObject;
import org.bouncycastle.asn1.DERUTF8String;

public class Main {

    private static final DERTaggedObject SEQ = new DERTaggedObject(
        false, 1,
        new DERSequence(new DERUTF8String("hello"), new DERUTF8String("world"))
    );

    private static final DERTaggedObject NESTED_SEQ = new DERTaggedObject(
        false, 1,
        new DERSequence(
            new DERSequence(new DERUTF8String("hello"), new DERUTF8String("world"))
        )
    );

    public static void main(String[] args) throws IOException {
        var seq = encodeDecode("seq.der", SEQ);
        var nestedSeq = encodeDecode("nested_seq.der", NESTED_SEQ);
    }

    static ASN1Encodable encodeDecode(String file, ASN1Encodable obj) throws IOException {
        var baos = new ByteArrayOutputStream();
        obj.toASN1Primitive().encodeTo(baos);
        var bytes = baos.toByteArray();

        Files.write(Paths.get(file), bytes);

        try (var is = new ASN1InputStream(new ByteArrayInputStream(bytes))) {
            return is.readObject();
        }
    }
}

I have two different tagged structures with implicit tags. Byte representation of SEQ and NESTED_SEQ objects seems to be ok:

$ xxd seq.der 
00000000: a10e 0c05 6865 6c6c 6f0c 0577 6f72 6c64  ....hello..world

$ openssl asn1parse -inform DER -in seq.der 
    0:d=0  hl=2 l=  14 cons: cont [ 1 ]        
    2:d=1  hl=2 l=   5 prim: UTF8STRING        :hello
    9:d=1  hl=2 l=   5 prim: UTF8STRING        :world

$ xxd nested_seq.der 
00000000: a110 300e 0c05 6865 6c6c 6f0c 0577 6f72  ..0...hello..wor
00000010: 6c64

$ openssl asn1parse -inform DER -in nested_seq.der 
    0:d=0  hl=2 l=  16 cons: cont [ 1 ]        
    2:d=1  hl=2 l=  14 cons: SEQUENCE          
    4:d=2  hl=2 l=   5 prim: UTF8STRING        :hello
   11:d=2  hl=2 l=   5 prim: UTF8STRING        :world

But when I decode these bytes back to java objects, then I got exactly same asn1 sequences despite that original objects was different:

If I understand correctly, no chances to distinguish between nested sequence with implicit tag and plain sequence with explicit tag (which represented by same byte sequences).

What can I do in such situation?

[peterdettman]

An ASN.1 parser cannot (in the general case) resolve a tag without external information. Especially implicit tagging requires to be told, at parse-time, what the correct tag actually is (implicit tagging elides the original tag). Parsing isn't really complete until all tagging has been resolved.

Example of "completing parsing" in this case:
var obj = encodeDecode("seq.der", SEQ);
var tag = ASN1TaggedObject.getInstance(encoded, BERTags.CONTEXT_SPECIFIC, 1); // Tag validation
var seq = ASN1Sequence.getInstance(tag, false); // ASN.1 types should usually provide a getInstance method like this

You'll see that this reconstructs the two different sequences correctly.

BTW, the two instances of DLTaggedObject are not the same: see the different values for explicitness. The parser is sometimes forced to guess whether it is dealing with an implicit tag or not. It records what it guessed in 'explicitness' and b/w that field and 'obj' it can reconstruct the data correctly once it has been told what the tagging should actually be.

I suggest you don't think of ASN1TaggedObject (and subclasses) as part of the data at all. Add them only when encoding and always resolve them as part of parsing (and don't blindly interpret the 'obj' field as the actual value).

If you are designing your own data formats here, I would suggest creating actual data types following the many examples of such in the library. You may also find that the small space savings from implicit tagging are not worth the extra confusion (YMMV).