Benchmark 389 Directory Server Performance with ldclt and logconv

Performance testing in 389 Directory Server spans three activities: building a reproducible dataset, driving repeatable client workloads, and analyzing what the server recorded. dsctl ldifgen prepares LDIF files; it does not issue sustained bind or search traffic. After the data is loaded, use ldclt or an equivalent multithreaded client for throughput measurements.

This guide covers benchmark planning, dataset preparation, validated loading, ldclt search and bind workloads, monitor deltas, nsslapd-statlog-level diagnosis, and logconv.pl reporting.

Before you start:

• Install 389 Directory Server — running instance (ldap1 on dc=example,dc=com in this lab)
• Generate test data with ldifgen — full generator reference
• dsctl commands and dsconf commands — import and monitoring
• Configure 389 Directory Server Indexes for Faster LDAP Searches — controlled before-and-after index changes

Tested on: Rocky Linux 10.2; 389 Directory Server 3.2.0.

Understand Performance Metrics

A useful benchmark answers one question at a time: exact-match search latency, bind rate, write throughput with plug-ins enabled, or the effect of one index change.

Distinguish these counters:

Attempted operations     what the client sent
LDAP operations completed what the server finished (monitor counters)
Expected-result hits    exact searches returning nentries=1
LDAP result-code errors non-zero err= values
Client failures         connection, TLS, or timeout errors

Client latency includes client processing, DNS, connection setup, TLS, authentication, network transit, server queuing, server processing, and response handling. etime is the server-side interval in the access log; it is not the same as complete client-observed latency. wtime, optime, and etime are related, but timing collection can make their sum differ slightly from etime.

High wtime:
Operations are waiting before execution.

High optime:
The server is spending time processing the operation.

High client latency with low etime:
The delay can be outside Directory Server, such as network or client handling.

Report minimum, median or p50, p95, p99, and maximum when your workload tool provides them. logconv.pl reports average wtime, optime, and etime across the analyzed window; pair those summaries with per-operation access-log detail or client-side histograms.

Plan a Reproducible Benchmark

Record the environment before the first run:

389 Directory Server version and package build
Operating system, CPU count, and available RAM
Storage type, filesystem, and container limits
Suffix, backend, entry count, and average entry size
Configured indexes, LMDB and cache settings, worker threads
Replication topology
TLS or clear-text LDAP
Client host CPU, memory, and network path
Load-generator process or thread count
Access-log and audit-log configuration
Page-cache state and Directory Server cache state
Connection model (persistent vs bind-each-operation)
Returned attribute list and average result size
Test duration, concurrency, warm-up plan, and exact commands

Example test matrix for a 100,000-user dataset:

Run each important scenario at least three times after warm-up. Report the median result and the spread between runs. Change only one server setting between baseline and comparison tests.

The load-generator host must have enough CPU, file descriptors, sockets, and network capacity to drive the server. If client CPU saturates first, you measured a client limit, not a Directory Server limit.

Prepare and Load the Test Dataset

Use a dedicated test instance, disposable suffix or backend, separate LDAP client host where possible, and benchmark-only credentials. Synchronize server and client clocks so access-log timestamps align with client measurements.

For generator details beyond the three datasets below, see Generate test data with ldifgen.

Datasets used in this guide

Always pin users under one parent so bind and search DNs stay predictable:

dsctl ldap1 ldifgen users \
    --suffix "dc=example,dc=com" \
    --parent "ou=People,dc=example,dc=com" \
    --number 100000 \
    --generic \
    --start-idx 1 \
    --ldif-file /tmp/users.ldif

A successful run ends with Successfully created LDIF file: /tmp/users.ldif. Confirm the file exists before import.

Inspect the first generated UID before building filters. In this lab, --start-idx 1 with --parent produced uid=user002, not uid=user1:

awk '/^uid: / {print $2; exit}' /tmp/users.ldif

Sample output:

user002

That value becomes the filter in your first exact-match smoke test.

Count user entries and estimate on-disk size:

grep -c '^dn: uid=' /tmp/users.ldif

Sample output for a 100,000-user file:

100000

du -h /tmp/users.ldif

On a 10,000-user file with the same options in this lab:

15M	/tmp/bench/users10k.ldif

That scales to roughly 150 MiB for 100,000 users (15M × 10). Measure size with df and du (du -h) on your host before import.Measure du -h on your host before import.

The generic generator creates predictable numbered UIDs. Padding and the first index depend on --start-idx and the installed package; inspect the first few dn: lines instead of assuming uid=user1.

Group data for membership benchmarks:

dsctl ldap1 ldifgen groups example_group__ \
    --number 500 \
    --suffix "dc=example,dc=com" \
    --parent "ou=groups,dc=example,dc=com" \
    --num-members 100 \
    --create-members \
    --member-parent "ou=People,dc=example,dc=com" \
    --member-attr member \
    --ldif-file /tmp/groups.ldif

A successful run ends with Successfully created LDIF file: /tmp/groups.ldif. With --create-members, the groups file also contains member user entries and may repeat suffix or container records. Prefer a clean test backend, or strip duplicate container dn: records before loading a second generated file.

Build several group shapes for membership benchmarks:

1,000 groups × 10 members
500 groups × 100 members
100 groups × 1,000 members

Mixed write workload for add, modify, rename, and delete throughput:

dsctl ldap1 ldifgen mod-load \
    --num-users 1000 \
    --create-users \
    --parent "ou=People,dc=example,dc=com" \
    --mod-attrs "sn" \
    --add-users 10 \
    --modrdn-users 100 \
    --del-users 100 \
    --delete-users \
    --ldif-file /tmp/modifications.ldif

Set --mod-users no higher than --num-users, or ldapmodify will fail when it reaches entries that were never created.

Choose the load path

Online ldapadd

Generate a PerfBench dataset rooted at the benchmark organizational unit:

dsctl ldap1 ldifgen users \
    --suffix "dc=example,dc=com" \
    --parent "ou=PerfBench,dc=example,dc=com" \
    --number 200 \
    --generic \
    --start-idx 1 \
    --ldif-file /tmp/perfbench-users.ldif

Sample output:

Writing LDIF ...
Successfully created LDIF file: /tmp/perfbench-users.ldif

The file includes ou=PerfBench plus 200 uid= entries. Add the organizational unit first when it does not already exist:

New entries in this section are added with the ldapadd command.

awk '/^dn: ou=PerfBench/ {print; p=1; next} p && /^$/ {print; p=0; next} p' \
    /tmp/perfbench-users.ldif |
    ldapadd -x \
        -H ldap://ldap1.example.com:389 \
        -D "cn=Directory Manager" \
        -y /root/dm.pw

Sample output:

adding new entry "ou=PerfBench,dc=example,dc=com"

For timing tests, load only the user entries. This awk command preserves complete multi-line LDIF records:

awk '/^dn: uid=user/ {print; p=1; next} p && /^$/ {print; p=0; next} p' \
    /tmp/perfbench-users.ldif > /tmp/perfbench-users-only.ldif

Confirm the extracted file before import:

grep -c '^dn: uid=' /tmp/perfbench-users-only.ldif

Sample output:

200

Time the online add workload:

/usr/bin/time -f 'elapsed=%e user=%U sys=%S' \
    ldapadd -x \
    -H ldap://ldap1.example.com:389 \
    -D "cn=Directory Manager" \
    -y /root/dm.pw \
    -f /tmp/perfbench-users-only.ldif

Sample output:

adding new entry "uid=user201,ou=PerfBench,dc=example,dc=com"
elapsed=24.03 user=0.02 sys=0.02

Two hundred attempted adds completed with zero failures in 24.03 seconds, or about 8.3 adds per second (200 ÷ 24.03).

Avoid -c on a clean benchmark unless you also count and report every error code.

Online backend import

Create a disposable backend for isolated import tests:

dsconf ldap1 backend create --suffix "dc=perftest,dc=example" --be-name perftest --create-suffix

Sample output:

The database was sucessfully created

Generate an LDIF that matches the disposable suffix:

dsctl ldap1 ldifgen users \
    --suffix "dc=perftest,dc=example" \
    --parent "ou=People,dc=perftest,dc=example" \
    --number 500 \
    --generic \
    --start-idx 1 \
    --ldif-file /tmp/perftest-users.ldif

Sample output:

Writing LDIF ...
Successfully created LDIF file: /tmp/perftest-users.ldif

Confirm the user count before import:

grep -c '^dn: uid=' /tmp/perftest-users.ldif

Sample output:

500

Copy the matching LDIF, set ownership, and restore SELinux context so the dirsrv user can read the file:

cp /tmp/perftest-users.ldif /var/lib/dirsrv/slapd-ldap1/ldif/perftest-users.ldif
chown dirsrv:dirsrv /var/lib/dirsrv/slapd-ldap1/ldif/perftest-users.ldif
chmod 600 /var/lib/dirsrv/slapd-ldap1/ldif/perftest-users.ldif
restorecon -v /var/lib/dirsrv/slapd-ldap1/ldif/perftest-users.ldif

These preparation commands exit silently when they succeed.

Run the import against perftest, not production userroot:

dsconf ldap1 backend import perftest /var/lib/dirsrv/slapd-ldap1/ldif/perftest-users.ldif

Sample output:

The import task has finished successfully

A successful online import reports that line and an exit code of 0 in the error log.

Inspect the error log and service journal:

tail -n 5 /var/log/dirsrv/slapd-ldap1/errors

Sample output:

import perftest: Import complete.  Processed 508 entries in 3 seconds. (169.33 entries/sec)
import perftest: Finished importing task ... Exit code is 0

The processed-entry count should match the LDIF plus any container records the file creates. In this lab, 500 user entries plus eight container records produced 508 processed entries.

journalctl -u dirsrv@ldap1 --since "5 minutes ago" --no-pager | tail -3

Sample output:

perftest: Finished importing task 'cn=import_2026-07-18t05:41:00.043494,cn=import,cn=tasks,cn=config'. Exit code is 0

Offline dsctl ldif2db

Stop the instance before an offline bulk import:

dsctl ldap1 stop

Sample output:

Instance "ldap1" has been stopped

Import the matching LDIF into the disposable backend:

dsctl ldap1 ldif2db perftest /var/lib/dirsrv/slapd-ldap1/ldif/perftest-users.ldif

Sample output:

ldif2db successful

Start the instance again:

dsctl ldap1 start

Sample output:

Instance "ldap1" has been started

The error log records processed entry counts:

import perftest: Import complete.  Processed 508 entries in 0 seconds.
ldbm_back_ldif2ldbm - Ending Off-line import of ldif file ... over backend perftest (rc=0).

rc=0 confirms the offline import completed without error.

Validate the Dataset and Test Accounts

Compare the generated file with the loaded directory:

EXPECTED=$(grep -c '^dn: uid=' /tmp/perfbench-users-only.ldif)

Record how many user entries the LDIF should create.

Directory queries in this section use the ldapsearch command.

ACTUAL=$(ldapsearch -LLL -x \
    -H ldap://ldap1.example.com:389 \
    -D "cn=Directory Manager" \
    -y /root/dm.pw \
    -b "ou=PerfBench,dc=example,dc=com" \
    "(objectClass=inetOrgPerson)" dn | grep -c '^dn:')

Count live inetOrgPerson entries under the benchmark organizational unit.

printf 'Expected: %s\nActual: %s\n' "$EXPECTED" "$ACTUAL"

Sample output:

Expected: 200
Actual: 200

Matching counts confirm the load finished before you start throughput tests.

Verify one exact UID from the generated file, not a broad wildcard:

FIRST_UID=$(awk '/^uid: / {print $2; exit}' /tmp/perfbench-users-only.ldif)
printf 'First UID: %s\n' "$FIRST_UID"

Sample output:

First UID: user002

Look up that UID to confirm the entry shape:

ldapsearch -LLL -x \
    -H ldap://ldap1.example.com:389 \
    -D "cn=Directory Manager" \
    -y /root/dm.pw \
    -b "ou=PerfBench,dc=example,dc=com" \
    "(uid=${FIRST_UID})" dn uid cn

Sample output:

dn: uid=user002,ou=PerfBench,dc=example,dc=com
uid: user002
cn: user002

One matching dn: line confirms the exact filter resolves to a real entry.

Confirm a generated user can bind before load testing. Create the benchmark directory and password file:

install -d -m 700 /tmp/bench printf '%s\n' 'user002' > /tmp/bench/user002.pw chmod 600 /tmp/bench/user002.pw

The generic account used in this tested dataset has user002 as its lab password. Confirm the generated userPassword value in your LDIF if your package produces a different value.

ldapwhoami -x \
    -H ldap://ldap1.example.com:389 \
    -D "uid=user002,ou=PerfBench,dc=example,dc=com" \
    -y /tmp/bench/user002.pw

Sample output:

dn: uid=user002,ou=perfbench,dc=example,dc=com

The returned DN confirms simple bind succeeded for a generated account.

A broad filter such as (uid=user*) matches user002, user050, and every other UID beginning with user. Use an equality filter such as (uid=user050) when validating one generated entry. Do not benchmark while imports, reindexing, fixup tasks, or replication initialization are still running.

Run Search, Bind and Write Workloads

Separate bind, search, and write scenarios. Do not combine filters with very different selectivity into one throughput number. Run warm-up operations before the measured interval; exclude warm-up from logconv time windows and client timers.

Serial connection, bind, and search smoke test

A shell loop that calls ldapsearch once per iteration measures end-to-end client latency, not isolated Directory Server search processing. Each iteration starts a new process, resolves the hostname, opens a connection, binds, sends one search, and exits.

/usr/bin/time -f 'elapsed=%e' sh -c '
ATTEMPTED=0; OK=0; FAIL=0
for TESTUID in user002 user050 user100; do
    ATTEMPTED=$((ATTEMPTED+1))
    OUT=$(ldapsearch -LLL -x \
        -H ldap://ldap1.example.com:389 \
        -D "cn=Directory Manager" \
        -y /root/dm.pw \
        -b "ou=PerfBench,dc=example,dc=com" \
        "(uid=${TESTUID})" dn)
    if echo "$OUT" | grep -c "^dn:" | grep -qx 1; then OK=$((OK+1)); else FAIL=$((FAIL+1)); fi
done
printf "attempted=%s ok=%s fail=%s\n" "$ATTEMPTED" "$OK" "$FAIL"
'

Sample output:

elapsed=0.53
attempted=3 ok=3 fail=0

All three exact searches returned one entry (ok=3). Ten serial searches that each open a new connection took about 1.31 seconds in a longer run in this lab. Use this pattern only as a functional smoke test, not as throughput measurement.

Concurrent exact searches with ldclt

Confirm ldclt is installed:

command -v ldclt

Sample output:

/usr/bin/ldclt

rpm -qf "$(command -v ldclt)"

Sample output:

389-ds-base-3.2.0-8.el10_2.x86_64

ldclt 2>&1 | head -1

Sample output:

ldclt version 4.23

Exact-match search workload with eight threads and up to 800 operations per thread (8 × 800 = 6,400 operations):

ldclt -h ldap1.example.com -p 389 \
    -D "cn=Directory Manager" \
    -w "$(cat /root/dm.pw)" \
    -b "ou=PerfBench,dc=example,dc=com" \
    -f "(uid=user002)" \
    -e esearch \
    -n 8 -T 800

Sample output:

ldclt[77373]: Global average rate:  800.00/thr  (320.00/sec), total:   6400
ldclt[77373]: Global no error occurs during this session.
ldclt[77373]: Exit status 0 - No problem during execution.

In 800.00/thr (320.00/sec), each thread completed an average of 800 operations, while all threads together averaged 320 operations per second. With -e esearch, ldclt binds once per thread and reuses that authenticated connection for subsequent searches. Exit status 0 means ldclt completed without a fatal condition. Confirm that the run recorded no LDAP errors by also checking for Global no error occurs during this session. If ldclt prints Global error lines, report those errors even when the command finishes.

For a fixed-duration test instead of a fixed operation count, use -N to set the number of ten-second samples. For example, -n 8 -N 60 runs approximately ten minutes.

Random exact-UID searches across the generated population:

ldclt -h ldap1.example.com -p 389 \
    -D "cn=Directory Manager" \
    -w "$(cat /root/dm.pw)" \
    -b "ou=PerfBench,dc=example,dc=com" \
    -f "(uid=userXXX)" \
    -e esearch,random,attrlist=uid:cn \
    -r 2 -R 201 \
    -n 8 -T 400

Sample output:

ldclt[77516]: Global average rate:  400.00/thr  (160.00/sec), total:   3200
ldclt[77516]: Global no error occurs during this session.
ldclt[77516]: Exit status 0 - No problem during execution.

The X placeholders in the filter pattern combine with -e esearch,random and -r/-R to pick a random UID suffix on each operation. Keep the requested attributes identical between comparison runs. For example, add attrlist=uid:cn to the execution parameters:

-e esearch,random,attrlist=uid:cn

Do not use -I for this purpose; -I configures LDAP result codes that ldclt should ignore.

No LDAP errors does not prove that every filter matched one entry. Before the benchmark, verify that the first and last generated UIDs exist. During the measured log window, confirm that the corresponding RESULT records report nentries=1 for exact-UID searches.

A simple preflight check:

for TESTUID in user002 user201; do
    COUNT=$(ldapsearch -LLL -x \
        -H ldap://ldap1.example.com:389 \
        -D "cn=Directory Manager" \
        -y /root/dm.pw \
        -b "ou=PerfBench,dc=example,dc=com" \
        "(uid=${TESTUID})" dn |
        grep -c '^dn:')

    printf '%s matches=%s\n' "$TESTUID" "$COUNT"
done

Sample output:

user002 matches=1
user201 matches=1

Exit status 0 means ldclt completed without a fatal condition. Confirm that the run recorded no LDAP errors by also checking for Global no error occurs during this session.

Application-account search workload with uid=user1 under ou=people (not Directory Manager):

ldclt -h ldap1.example.com -p 389 \
    -D "uid=user1,ou=people,dc=example,dc=com" \
    -w "$(cat /root/user1.pw)" \
    -b "ou=people,dc=example,dc=com" \
    -f "(uid=user1)" \
    -e esearch \
    -n 4 -T 200

Sample output:

ldclt[71092]: Global average rate:  200.00/thr  ( 40.00/sec), total:    800
ldclt[71092]: Global no error occurs during this session.
ldclt[71092]: Exit status 0 - No problem during execution.

Production-like tests should use an application account and the same TLS mode your clients use. Exit status 0 means ldclt completed without a fatal condition; confirm Global no error occurs during this session. before treating the run as error-free.

Create a client-side NSS database that trusts the Directory Server CA, then point -Z at a certificate database file inside that directory:

mkdir -p /root/ldcltnss
certutil -N -d sql:/root/ldcltnss -f /dev/null
certutil -A -d sql:/root/ldcltnss -n "389ds-ca" -t "CT,," \
    -i /etc/dirsrv/slapd-ldap1/ca.crt
certutil -L -d sql:/root/ldcltnss

Sample output:

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

389ds-ca                                                     CT,,

LDAPS throughput workload using the client NSS database:

ldclt -h ldap1.example.com -p 636 \
    -Z /root/ldcltnss/cert9.db \
    -D "cn=Directory Manager" \
    -w "$(cat /root/dm.pw)" \
    -b "ou=PerfBench,dc=example,dc=com" \
    -f "(uid=user002)" \
    -e esearch \
    -n 4 -T 80

Sample output:

ldclt[77550]: Global average rate:   80.00/thr  ( 16.00/sec), total:    320
ldclt[77550]: Global no error occurs during this session.
ldclt[77550]: Exit status 0 - No problem during execution.

This workload creates one TLS connection per thread and reuses it. The handshake is amortized across that thread's operations. To measure handshake-per-operation cost, run a separate reconnect or bind-each-operation scenario. Compare LDAPS only against another LDAPS run, not against a persistent clear-text connection. Exit status 0 means ldclt completed without a fatal condition; confirm Global no error occurs during this session. before treating the run as error-free.

Use separate scenarios for presence, prefix, middle substring, compound filters, and group membership searches.

Bind workloads

Verify one generated account binds:

ldapwhoami -x \
    -H ldap://ldap1.example.com:389 \
    -D "uid=user002,ou=PerfBench,dc=example,dc=com" \
    -y /tmp/bench/user002.pw

Sample output:

dn: uid=user002,ou=perfbench,dc=example,dc=com

Single-account concurrent bind throughput with bindeach,bindonly (one bind per operation on uid=user002):

ldclt -h ldap1.example.com -p 389 \
    -b "ou=PerfBench,dc=example,dc=com" \
    -e bindonly,bindeach \
    -D "uid=user002,ou=PerfBench,dc=example,dc=com" \
    -w user002 \
    -n 4 -T 80

Sample output:

ldclt[77535]: Global average rate:   80.00/thr  (  6.40/sec), total:    320
ldclt[77535]: Global no error occurs during this session.
ldclt[77535]: Exit status 0 - No problem during execution.

The lower aggregate rate reflects the cost of opening a connection and binding on every operation. This scenario measures repeated authentication of one account. A workload intended to represent binds distributed across the complete generated population should use randomized bind DNs and matching passwords as a separate scenario. Use disposable test credentials only. Exit status 0 means ldclt completed without a fatal condition; confirm Global no error occurs during this session. before treating the run as error-free.

Write throughput

Generate a small mixed file for operation counting:

dsctl ldap1 ldifgen mod-load \
    --num-users 10 \
    --create-users \
    --parent "ou=WriteBench,dc=example,dc=com" \
    --mod-attrs "sn" \
    --add-users 1 \
    --mod-users 8 \
    --modrdn-users 1 \
    --del-users 1 \
    --ldif-file /tmp/bench/mod-small.ldif

Sample output:

Writing LDIF ...
Successfully created LDIF file: /tmp/bench/mod-small.ldif

Count each changetype before running a mixed file:

grep -E '^changetype:' /tmp/bench/mod-small.ldif | sort | uniq -c

Sample output:

11 changetype: add
      1 changetype: delete
      8 changetype: modify
      1 changetype: modrdn

The counts tell you how many of each operation type the benchmark file will attempt. Keep this mixed file for illustrating operation counting. Continue using the separately generated write-clean.ldif for the actual timed add-and-modify benchmark.

For a clean add-and-modify benchmark, set every operation count explicitly and disable rename and delete operations:

dsctl ldap1 ldifgen mod-load \
    --num-users 10 \
    --create-users \
    --parent "ou=WriteBench,dc=example,dc=com" \
    --mod-attrs "sn" \
    --add-users 0 \
    --mod-users 10 \
    --modrdn-users 0 \
    --del-users 0 \
    --ldif-file /tmp/bench/write-clean.ldif

Sample output:

Writing LDIF ...
Successfully created LDIF file: /tmp/bench/write-clean.ldif

Confirm the file contains only add and modify records before you benchmark:

grep -E '^changetype:' /tmp/bench/write-clean.ldif | sort | uniq -c

Sample output:

10 changetype: add
     10 changetype: modify

Create the parent organizational unit if it does not exist:

ldapadd -x \
    -H ldap://ldap1.example.com:389 \
    -D "cn=Directory Manager" \
    -y /root/dm.pw <<'EOF'
dn: ou=WriteBench,dc=example,dc=com
objectClass: organizationalUnit
ou: WriteBench
EOF

Sample output:

adding new entry "ou=WriteBench,dc=example,dc=com"

Apply the workload and time the run:

Entry updates in this section use the ldapmodify command.

/usr/bin/time -f 'elapsed=%e' ldapmodify -x \
    -H ldap://ldap1.example.com:389 \
    -D "cn=Directory Manager" \
    -y /root/dm.pw \
    -f /tmp/bench/write-clean.ldif

Sample output:

modifying entry "uid=user01,ou=WriteBench,dc=example,dc=com"
elapsed=5.10

In this lab the file contained 10 adds and 10 modifies with zero LDAP errors in 5.10 seconds, or about 3.9 total operations per second. Reset the test subtree to the same initial state before every repetition; reapplying an LDIF that contains add operations fails once the entries already exist. A mixed file that includes modrdn and delete records can fail partway through (ldap error 34 on rename in this lab) while still reporting elapsed time, so split operation types or count successes and failures explicitly.

Prefer separate LDIF files per operation type when you need independent add, modify, rename, and delete rates. Record which plug-ins are enabled; MemberOf, Referential Integrity, and related plug-ins add expected consistency cost.

In a replicated topology, measure local supplier write completion and the time until changes are visible on required consumers. Otherwise the write rate does not describe end-to-end consistency.

Cold Directory Server cache and warm Directory Server cache

Restart clears internal entry and DN caches, not necessarily the Linux page cache:

dsctl ldap1 restart

Sample output:

Instance "ldap1" has been restarted

Run the first measured workload immediately for a cold Directory Server cache result. Repeat the identical workload until dsconf ldap1 monitor dbmon cache hit ratios stabilize for a warm Directory Server cache result. Label results precisely; a restart-only test is not a controlled cold OS page-cache test unless you prepare page cache separately.

Record total entries, database size, index size, LMDB map usage, free disk space, and replication state before benchmarking. Do not start while imports, reindexing, fixup tasks, or replication initialization are still running.

Capture Server, Client and OS Metrics

dsconf monitor server counters are cumulative. Save a baseline before the workload:

dsconf ldap1 monitor server | grep -E '^(opsinitiated|opscompleted|totalconnections):' \
    > /tmp/server-before.txt

Run the workload, then capture the same fields again:

dsconf ldap1 monitor server | grep -E '^(opsinitiated|opscompleted|totalconnections):' > /tmp/server-after.txt

Compare the two snapshots side by side:

paste /tmp/server-before.txt /tmp/server-after.txt

Sample output:

opsinitiated: 17648	opsinitiated: 24069
opscompleted: 17647	opscompleted: 24068
totalconnections: 1100	totalconnections: 1109

During an eight-thread ldclt search run of about twenty seconds, opscompleted rose by 6,421 operations, or roughly 321 server-side completions per second. Pair monitor deltas with ldclt client totals; one client operation does not always equal one server opscompleted increment.

Also save dsconf ldap1 monitor dbmon before and after for entry and DN cache hit ratios:

dsconf ldap1 monitor dbmon | grep -E 'Backends:|dc=|Hit Ratio|Cache Count' > /tmp/dbmon-before.txt

Run the workload, then capture the same fields again:

dsconf ldap1 monitor dbmon | grep -E 'Backends:|dc=|Hit Ratio|Cache Count' > /tmp/dbmon-after.txt

Sample output from the before snapshot:

- Cache Hit Ratio:     97%
Backends:
  - dc=perftest,dc=example (perftest):
    - Entry Cache Hit Ratio:        51%
    - Entry Cache Count:            1
  - dc=example,dc=com (userroot):
    - Entry Cache Hit Ratio:        73%
    - Entry Cache Count:            762

The first Cache Hit Ratio line is the normalized DN cache. Each backend block reports its own entry-cache statistics. After the workload, the DN cache hit ratio rose from 96% to 97% in this lab. Rising hit ratios during warm-up runs confirm the cache is filling.

On the Directory Server host, install sysstat when iostat and pidstat are missing. On RHEL-family systems:

dnf install -y sysstat

On Debian, Ubuntu, or SUSE, install the equivalent sysstat package for your distribution.

iostat and pidstat become available after the package installs.

Start background collectors before the workload. The trap stops them if the shell exits early:

DS_PID=$(systemctl show --property MainPID --value dirsrv@ldap1)
test "$DS_PID" -gt 0 || { echo "ldap1 is not running" >&2; exit 1; }

vmstat 1 > /tmp/vmstat.txt &
VMSTAT_PID=$!
iostat -xz 1 > /tmp/iostat.txt &
IOSTAT_PID=$!
pidstat -dur -p "$DS_PID" 1 > /tmp/pidstat.txt &
PIDSTAT_PID=$!

cleanup_metrics() {
    kill "$VMSTAT_PID" "$IOSTAT_PID" "$PIDSTAT_PID" 2>/dev/null
}
trap cleanup_metrics EXIT INT TERM

Run the measured workload, then stop the collectors explicitly:

cleanup_metrics
trap - EXIT INT TERM

After the workload, inspect the completed files. The last lines summarize CPU, I/O wait, and ns-slapd disk writes during the test:

head -n 2 /tmp/vmstat.txt
tail -n 2 /tmp/vmstat.txt

Sample output:

procs -----------memory---------- ---swap-- -----io---- -system-- -------cpu-------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st gu
 0  0 342084 243184  97476 1285532    0    0     0  7716 2810 6605  8 62 28  3  0  0
 2  1 342084 243188  97476 1285532    0    0     0  9204 2466 1152  1 47 47  6  0  0

The wa column is I/O wait; high values under load suggest storage contention.

tail -n 10 /tmp/iostat.txt

Sample output:

Device            r/s     rkB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wkB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dkB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
sda             34.00   1548.00    16.00  32.00    2.18    45.53   83.00   1108.00    58.00  41.13    1.41    13.35    0.00      0.00     0.00   0.00    0.00     0.00   36.00    1.08    0.23  15.00

Elevated w_await on the database disk points to slow writes. Column order can differ between sysstat versions; use the header row in the saved file to locate the field you need.

tail -n 2 /tmp/pidstat.txt

Sample output:

Average:      UID       PID   kB_rd/s   kB_wr/s kB_ccwr/s iodelay  Command
Average:      389     70786      0.00  10669.33      0.00       0  ns-slapd

High kB_wr/s on ns-slapd during a read benchmark can mean logging or cache write-back, not necessarily client write load.

Analyze Access Logs with logconv.pl

Keep access-log buffering enabled. Buffered events can appear in the file slightly after they occur; allow the buffer to flush before the final logconv.pl pass.

For focused search diagnosis, enable per-search index statistics without restarting in this lab:

dsconf ldap1 config replace nsslapd-statlog-level=1

Confirm the setting took effect:

dsconf ldap1 config get nsslapd-statlog-level

Sample output:

nsslapd-statlog-level: 1

Run the diagnostic search workload, then restore the default:

dsconf ldap1 config replace nsslapd-statlog-level=0

Confirm the default returned:

dsconf ldap1 config get nsslapd-statlog-level

Sample output:

nsslapd-statlog-level: 0

Confirm behaviour on your installed release; do not restart automatically unless the local package requires it.

A correlated SRCH, STAT, and RESULT sequence from this lab (same conn and op):

Sample output:

[18/Jul/2026:04:59:05.486992148 +0530] conn=41 op=1 SRCH base="ou=PerfBench,dc=example,dc=com" scope=2 filter="(uid=user050)" attrs="distinguishedName"
[18/Jul/2026:04:59:05.488080645 +0530] conn=41 op=1 STAT read index: attribute=uid key(eq)=user050 --> count 2 (duration 0.000009544)
[18/Jul/2026:04:59:05.488088842 +0530] conn=41 op=1 RESULT err=0 tag=101 nentries=1 wtime=0.000131376 optime=0.001091376 etime=0.001220656

STAT shows which index served the filter and how many candidates were considered.

Summarize the benchmark window:

logconv.pl /var/log/dirsrv/slapd-ldap1/access

Sample output:

Peak Concurrent Connections:   3
Total Operations:              52132
Searches:                      26509         (1.01/sec)
Binds:                         820           (0.03/sec)
Average wtime (wait time):     0.000299044
Average optime (op time):      0.011726487
Average etime (elapsed time):  0.012022403

Use the average timing lines to spot queueing (wtime) versus processing cost (optime).

List files before analyzing a wildcard glob, and decompress rotated logs first if your logconv.pl build cannot read .gz files directly:

logconv.pl -bc /var/log/dirsrv/slapd-ldap1/access*

-b prints bind statistics. -c prints connection-code statistics.

Sample output:

----- Total Connection Codes -----
U1      984   Cleanly Closed Connections
B1       46   Bad Ber Tag Encountered

----- Top 20 Bind DN's -----
1570            cn=directory manager
427             uid=user002,ou=perfbench,dc=example,dc=com

U1 is a clean connection close. The bind-DN list shows which accounts dominated the test window. For every bind DN, logconv.pl -B ALL ... is available on recent builds.

Restrict analysis to the measured interval:

logconv.pl \
    -S "[18/Jul/2026:04:58:00.000000000 +0530]" \
    -E "[18/Jul/2026:04:59:30.000000000 +0530]" \
    /var/log/dirsrv/slapd-ldap1/access*

-S and -E limit the report to the benchmark window so warm-up traffic does not dilute the result.

Per-minute CSV output:

logconv.pl -M /tmp/access-per-minute.csv /var/log/dirsrv/slapd-ldap1/access

Inspect the header and first data rows:

head -n 3 /tmp/access-per-minute.csv

Sample output:

Time,time_t,Results,Search,Add,Mod,Modrdn,Moddn,Compare,Delete,Abandon,Connections,SSL Conns,Bind,Anon Bind,Unbind,Unindexed search,Unindexed component,Invalid filter,ElapsedTime
17/Jul/2026:21:25:43.943589500 +0530,1784303700,3,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0.062165106
17/Jul/2026:21:35:19.781585700 +0530,1784304300,3,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0.002444701

The operation-count columns and reported ElapsedTime value help compare activity across each generated time interval.

Slow unindexed searches (-T sets an elapsed-time threshold for reporting unindexed searches; -u prints detail):

logconv.pl -T 0.5 -u /var/log/dirsrv/slapd-ldap1/access*

Aggregate unindexed component counts:

logconv.pl -U /var/log/dirsrv/slapd-ldap1/access*

Sample output:

Unindexed Searches:            0
Unindexed Components:          4007
  -  Unindexed Filters:   Filter:   (uid=user*) (occurrences 3202)
                          - Bind DN:  cn=directory manager (binds  3202)

A high count on (uid=user*) here matches a repeated broad substring workload, not a random exact-UID workload with -e esearch,random.

For uncompressed logs only, grep can find individual notes=A or notes=U results. Use zgrep on compressed rotated files:

grep -E 'notes=.*(A|U)' /var/log/dirsrv/slapd-ldap1/access | head -n 1

Sample output:

[17/Jul/2026:21:48:01.397885026 +0530] conn=198 op=1 RESULT err=0 tag=101 nentries=5 wtime=0.000210469 optime=0.037150878 etime=0.037353849 notes=U details="Partially Unindexed Filter"

notes=U flags a partially unindexed filter; correlate with STAT lines before adding indexes.

Do not add an index for every notes=U event. The ID-list scan threshold can produce the same note even when an index exists. Use search limits and index configuration when you move from measurement to tuning.

Compare and Interpret Results

Controlled change sequence:

Run baseline
        ↓
Add one equality index
        ↓
Rebuild the index
        ↓
Warm the Directory Server cache
        ↓
Repeat the identical workload
        ↓
Compare results

Do not disable required consistency or security plug-ins merely to obtain a better benchmark number.

Clean Up and Preserve Results

Before deleting test data, retain the test plan, exact commands, generated LDIF files, server configuration, monitor snapshots, access logs, logconv.pl output, operating-system metrics, and results tables.

Remove test data with one controlled method:

• Delete the disposable organizational unit or suffix
• Restore the test backend from backup
• Reimport baseline LDIF into a disposable backend
• Recreate the test instance

Verify cleanup removed the test organizational unit from the parent suffix:

COUNT=$(ldapsearch -LLL -x \
    -H ldap://ldap1.example.com:389 \
    -D "cn=Directory Manager" \
    -y /root/dm.pw \
    -b "dc=example,dc=com" \
    "(ou=PerfBench)" dn |
    grep -c '^dn:')

printf 'PerfBench OU matches remaining: %s\n' "$COUNT"

Sample output after successful cleanup:

PerfBench OU matches remaining: 0

Before cleanup, the same command reports 1 when only the ou=PerfBench container remains. Because LDAP child entries cannot remain under a deleted parent DN, zero matches confirms that the benchmark OU was removed. This check is intended for complete subtree deletion, not for cleanup that retains the OU. A subtree search against a deleted organizational unit can return LDAP result code 32 (No such object). Searching from the parent suffix avoids that ambiguity.

Confirm temporary settings are restored:

nsslapd-statlog-level=0
Temporary indexes removed
Resource limits restored
Background metric collectors stopped
Disposable password and LDIF files removed

Benchmark checklist

1. Define one measurable question and success criteria.
2. Record server, client, dataset, and connection model.
3. Generate LDIF with fixed --parent and inspect the first UID.
4. Load through the path that matches the question; treat import as destructive.
5. Verify expected versus actual entry counts and one exact bind.
6. Warm up, then run ldclt workloads and capture monitor deltas.
7. Analyze the measured time range with logconv.pl.
8. Change one setting, repeat, and compare median results across at least three runs.
9. Preserve raw artifacts and clean up the test subtree.

What's next

After you complete this guide, continue with:

• Tune LMDB, Entry Cache and DN Cache in 389 Directory Server — LMDB cache and map-size tuning
• Configure LDAP Search Limits in 389 Directory Server — lookthrough and ID-list limits
• Configure LDAP Threads and Connections in 389 Directory Server — worker thread configuration
• Tune Large LDAP Groups and memberOf Performance in 389 Directory Server — large static group behavior