United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
JDK-8179389 : X509Certificate generateCRLs is extremely slow using a PEM crl list

Details
Type:
Bug
Submit Date:
2017-04-26
Status:
Resolved
Updated Date:
2017-05-23
Project Name:
JDK
Resolved Date:
2017-05-10
Component:
security-libs
OS:
linux
Sub-Component:
java.security
CPU:
x86_64
Priority:
P4
Resolution:
Fixed
Affected Versions:
8,9
Fixed Versions:
10 (b08)

Related Reports

Sub Tasks

Description
FULL PRODUCT VERSION :
java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b14)
OpenJDK 64-Bit Server VM (build 25.121-b14, mixed mode)


ADDITIONAL OS VERSION INFORMATION :
Linux 4.10.9-200.fc25.x86_64 #1 SMP Mon Apr 10 14:48:16 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
Loading a PEM CRL list with 110 CRLs and 200MB in size is 25 minutes in my laptop. Using a DER list is 12 seconds.

The main reason is the X509Factory.java class uses a buffer of 2048 to store the base64 data and if it needs more it adds chucks by 1024 each (using Arrays.copyOf). That means to create a 10MB pem it calls to Arrays.copyOf 10000 times (allocate and copy). That's the reason to be painfully slow.

See here:
http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/sun/security/provider/X509Factory.java#l482



STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Just download a big CRL, convert to PEM and try the generateCRLs method. See the difference between PEM and DER.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Load the PEM crl in a reasonable time. My patch below does the load of 200MB in 83s and a 33MB CRL in 14s.
ACTUAL -
A 33MB CRL is 6 minutes in time to be loaded. A 200MB file with several CRLs is 25 minutes.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.File;
import java.io.FileInputStream;
import java.util.Collection;
import java.security.cert.Certificate;
import java.security.cert.CertificateFactory;

public class LoadCerts {

    public static void main(String[] args) throws Exception {
        if (args.length != 1) {
            throw new IllegalArgumentException("The first argument should be the PEM file.");
        }
        File f = new File(args[0]);
        if (!f.exists() || !f.canRead()) {
            throw new IllegalArgumentException(String.format("Invalid file %s", args[0]));
        }
        try (FileInputStream is = new FileInputStream(f)) {
            CertificateFactory cf = CertificateFactory.getInstance("X.509");
            long start = System.currentTimeMillis();
            Collection crls = cf.generateCRLs(is);
            long end = System.currentTimeMillis();
            System.out.println(String.format("Loaded %s certificates from %s in %d seconds.", crls.size(), args[0], (end - start) / 1000L));
        }
    }
}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Consider this patch, it uses the new java8 Base64.Decoder to just use an InputStream over the Base64 (avoiding intermediary buffers). It loads a 33MB crl in 14s instead of 6 minutes.

--- /home/rmartinc/jdk8/jdk/src/share/classes/sun/security/provider/X509Factory.java	2017-04-26 09:23:14.807876940 +0200
+++ X509Factory.java	2017-04-26 13:28:51.687191779 +0200
@@ -546,22 +546,17 @@
             }
 
             // Step 3: Read the data
-            while (true) {
-                int next = is.read();
-                if (next == -1) {
-                    throw new IOException("Incomplete data");
-                }
-                if (next != '-') {
-                    data[pos++] = (char)next;
-                    if (pos >= data.length) {
-                        data = Arrays.copyOf(data, data.length+1024);
-                    }
-                } else {
-                    break;
-                }
-            }
+            InputStream b64is = Base64.getMimeDecoder().wrap(is);
+            ByteArrayOutputStream bout = new ByteArrayOutputStream(2048);
+            c = b64is.read();
+            bout.write(c);
+            readBERInternal(b64is, bout, c);
 
             // Step 4: Consume the footer
+            c = is.read();
+            while (c != '-' && c != -1) {
+	        c = is.read();
+            }
             StringBuffer footer = new StringBuffer("-");
             while (true) {
                 int next = is.read();
@@ -575,7 +570,7 @@
 
             checkHeaderFooter(header.toString(), footer.toString());
 
-            return Base64.getMimeDecoder().decode(new String(data, 0, pos));
+            return bout.toByteArray();
         }
     }



                                    

Comments



Hardware and Software, Engineered to Work Together