Generational Garbage Collection with IBM JVM for Oracle Unified Directory

A main difference between IBM J9 JVM and Oracle JVM is that IBM’s JVM has different heap structure than Oracle’s JVM. By default IBM JVM (java 6) does not use Generational Garbage Collector. Such setting has adverse effects on OUD as OUD allocates a huge numbers of transient objects. Using the default IBM JVM settings lead to full garbage collections that freezes the directory services.

I recommend to use generational heap structure systematically with OUD. This can be enabled by the JVM option
–Xgcpolicy:gencon

OUD provides a means of configuring the Java Virtual Machine (JVM) and Java options for each command-line utility and for the directory server itself. The Java configuration is provided in a properties file, located at instance-dir/OUD/config/java.properties. The configuration specified in this file is taken into account each time the dsjavaproperties command is run. If you do not run the dsjavaproperties command, the properties file is ignored. More about JVM param settings at http://docs.oracle.com/cd/E29407_01/install.111200/e23737/configuring_jvm.htm

To use the gencon garbage collector policy for a specific OUD instance, do the following:

Edit the java.properties file located at instance-dir/OUD/config/java.properties
Add -Xgcpolicy:gencon at the end of the line starting with start-ds
Run instance-dir/OUD/bin/dsjavaproperties
Restart the OUD instance

OUD Database Indexes & Performance Tuning

Oracle Unified Directory (OUD) database indexes enable (search) directory requests  to be processed efficiently.

Indexes are files that contain lists of values, where each value is associated with a list of entry identifiers to suffixes in the directory server database. When the directory server processes a search request, it searches the database using the list of entry identifiers in the indexes, thus speeding up the search. If indexes did not exist, the directory server would have to look up each entry in the database, which dramatically degrades performance.

First of all, unindexed searches are rejected by default unless the (authenticated)  user has the privilege to perform them because unindexed searches  may negatively impact overall directory performances. In some circumstances, unindexed searches are a valid approach, so it is possible to explicitely  assign the unindexed-search privilege to a user or a group of user. A user attempting to perform an unindexed search gets the message “you do not have sufficient privileges to perform an unindexed search”.

Note that unindexed searches always complete sucessfully when the amount of entries in the database does not exceed the value if index-entry-limit (4000 by default), which is unlikely to be seen in real deployments.

The verify-index command can be used to check the consistency between the index and entry data within the directory server database. This command also provides information about the number of index keys that have reached the index entry limit (See below).

The rebuild-index command can be used to rebuild an index. It is useful in the following cases:

• When the index-entry-limit property of an index changes
• When a new index is created

The index-entry-limit tuning parameter (known as AllIDs threshold in Oracle Directory Server Entreprise Edition) controls the maximum numbers of entries kept in an index record as scaning the whole database may become a more efficient option when the number of entries associated with a given index value becomes huge.

Tuning indexes upfront may not be an obvious task, that’s the reason why OUD provides you with  both  static and  dynamic index analyser tools:

The static analyser is delivered as part of the dbtest tool.  The dbtest list-index-status -n <dbName> -b <dbSuffix> command let you figure out how many entries are associated with each index value, and tune index-entry-limit when needed..

The dynamic analyser can be used to monitor live search filters received by OUD and corresponding  index usage: Live index analysis must be first enabled at the database level. At that point, subsequent searches hitting OUD are analysed and the corresponding statistics are kept in memory.  Statistics can be retrieved from the in-memory suffix cn=monitor as part of the database environmenent entry.

To enable live index data collection, run  the following command:

$ dsconfig set-workflow-element-prop –element-name userRoot  –set index-filter-analyzer-enabled:true  –set max-entries:100 -h localhost -p 4444 -D cn=directory\ manager -w <password>

The set-max-entries parameter controls how many statistical records are kept in memory.

To retrieve the results, run the following search:

$ ldapsearch -p 1389 -D cn=directory\ manager -w <password>  -b “cn=<dbName> Database Environment,cn=monitor”  “(objectclass=*)”  filter-usefilter-use: (description=*) hits:2 maxmatches:-1 message:presence index type is disabled for the description attribute

Important note:  It is not recommanded to enable statistic collection in a production system for a long time as it consumes ressources.

It is also possible to use the operational attribute debugsearchindex to figure out which entries are targetted by each component of a LDAP search filter. This is a lightweight yet convenient way to debug index-related issues.

$ ./ldapsearch -p 11389 -b “o=example” -D “cn=directory manager” -w *** “(&(uid=user.9*)(objectclass=*))”debugsearchindex
dn: cn=debugsearch
debugsearchindex: filter=(&(objectClass=*)[NOT-INDEXED](uid=user.9*)[COUNT:111]) [COUNT:111] scope=wholeSubtree[LIMIT-EXCEEDED:4101] final=[COUNT:111]