When an LDAP server responds with many entries (>300,000) an LDAP client crashes with an "java.lang.OutOfMemoryError: Java heap space" if the request results cannot be processed fast enough.

According to the customer, the classes that cause the problem are found in the package com.sun.jndi.ldap. The class Connection creates a thread with an instance of itself. This thread runs the code found in the method run(). This thread reads incoming data from socket and stores them into a buffer in LdapRequest via the method "addReplyBer". The LdapRequest contains a buffer stored as a Vector in the instance variable "replies".  The class LdapRequest is also called by the Ldap Client via method "getReplyBer". This call is synchrone with the Ldap Client. The buffer is filled with by the producer in "addReplyBer" and emptied by a consumer "getReplyBer".

The problem is that the producer is faster than the consumer. This means that the buffer
in LdapRequest is filling up faster than it is emptied. This results in the problem than
a system will crash with a heap exception when large LDAP results are processed. When LDAP entries are retrieved and discarded without any processing, the LDAP client will not crash.

The parameter "BatchSize" has no effect on this problem, since this is only used by the
consumer thread.

Reproducing the problem:

Create a LDAP database with at least 300,000 entries. Then use this database with the test program below (you need to adjust some LDAP parameters in order to use it with your LDAP database).

--- begin ---
package ldaptest;

import java.io.FileOutputStream;
import java.io.ObjectOutputStream;
import java.util.Hashtable;

import javax.naming.Context;
import javax.naming.NamingEnumeration;
import javax.naming.directory.SearchControls;
import javax.naming.directory.SearchResult;
import javax.naming.ldap.InitialLdapContext;
import javax.naming.ldap.LdapContext;

public class Main
{
   public static LdapContext getDirContext() throws Exception
   {
       Hashtable<String, String> env = new Hashtable<String, String>();
       env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
       env.put(Context.PROVIDER_URL, "ldap://172.23.24.206:30000");
       
       env.put(Context.SECURITY_AUTHENTICATION, "simple");
       env.put(Context.SECURITY_PRINCIPAL, "cn=pg,dc=xxxxxx,dc=com");
       env.put(Context.SECURITY_CREDENTIALS, "xxxxx");
       env.put(Context.BATCHSIZE, "5");
       
       // Create the initial context
       LdapContext ctx = new InitialLdapContext(env, null);
       return ctx;
   }
   
   /**
    * @param args
    */
   public static void main(String[] args) throws Exception
   {
       System.out.println("All systems ready....");
       Thread.sleep(2000); // give profiler change to connect
       System.out.println("Go!");
       Runtime s_runtime = Runtime.getRuntime ();
       System.out.println("Memory totti:"+s_runtime.totalMemory());
       
       LdapContext ctx = Main.getDirContext();
       ctx.addToEnvironment("java.naming.ldap.derefAliases", "never");
       String dn = "ou=subscribers,dc= xxxxxx,dc=com";
       String filter = "ID=*";
       SearchControls ctls = new SearchControls();
       ctls.setTimeLimit(0);
       ctls.setSearchScope(SearchControls.SUBTREE_SCOPE);
       ctls.setCountLimit(0);
       ctls.setReturningAttributes(new String[] {"ID"});

       NamingEnumeration<SearchResult> answer = ctx.search(dn,filter, ctls);
             
       FileOutputStream fops = new FileOutputStream("./tmp.obj");
       ObjectOutputStream oops = new ObjectOutputStream(fops);
               
       long count = 0;
       try
       {
           while (answer.hasMore())
           {
               count++;
               SearchResult sr = (SearchResult) answer.next();
               oops.writeObject(sr); // processing is done here
           }
       }
       catch (Exception e)
       {
           e.printStackTrace();
       }
       oops.close();

       System.out.println("CCC:"+count);
       System.out.println("Totti:"+s_runtime.totalMemory());
   }
}
--- end ---

Tests with the program can be found in the comments section.