2564 certificates that fail to parse

[tomfitzhenry]

I'm parsing a large number of public certificates from Google's Certificate Transparency log servers, using hs-certificate. Of the 3.4 million certificates parsed so far, only 2,564 certificates have failed.

The 2,564 certificates and code to reproduce the failures is available at https://gist.github.com/tomfitzhenry/9124641 .

The certificates fail to parse for a variety of reasons. Here they are with the number of failures per group:

   2410 signed object error: "runParseASN1: remaining state [Start Sequence,OID [2,5,4,3],ASN1String (ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "domain-removed.com"}),End Sequence]"
    140 signed object error: "runParseASN1: remaining state [Start Sequence,OID [2,5,4,5],ASN1String (ASN1CharacterString {characterEncoding = Printable, getCharacterStringRawData = "domain-removed.com"}),End Sequence]"
      6 signed object error: "runParseASN1: remaining state [Start Sequence,OID [1,2,840,113549,1,9,2],ASN1String (ASN1CharacterString {characterEncoding = IA5, getCharacterStringRawData = "domain-removed.com"}),End Sequence]"
      3 signed object error: "runParseASN1: remaining state [Start Sequence,OID [2,5,4,7],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,8],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,3],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,11],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,10],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence]"
      3 signed object error: "runParseASN1: remaining state [Start Sequence,OID [2,5,4,7],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,8],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,3],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,10],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,11],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence]"
      1 signed object error: "runParseASN1: remaining state [Start Sequence,OID [2,5,4,7],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,8],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,11],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,3],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence,Start Sequence,OID [2,5,4,10],ASN1String (ASN1CharacterString {characterEncoding = UTF8, getCharacterStringRawData = "domain-removed.com"}),End Sequence]"
      1 signed object error: "fromASN1: X509.ExtensionRaw: OID=[2,5,29,32]: cannot decode data: StreamUnexpectedEOC"```

[vincenthz]

awesome tests. I fixed 2563 of those failures, and I'll have a look at the remaining one. Thanks a lot.

[tomfitzhenry]

No, thank you! Nice response time.

[vincenthz]

ok the remaining problem is a minor tweak in the ASN.1 parser, and has been fixed in asn1-encoding-v0.8.1.3. There's also now a x509-v1.4.8 that fix the 2563 other failures. I think all the decoding problem is solved now, don't hesitate to re-open if something is missing.