Restore a Replicated 389 Directory Server Safely

A failed replica is a replication-state problem, not only a database-restore problem. The instance can start successfully while its replica update vector (RUV), changelog, and agreements no longer match the rest of the topology.

This guide begins where the back up and restore chapter ends: a replicated server has failed and you must decide whether a backup can safely rejoin the topology or whether a healthy supplier should reinitialize the replica.

Before you start:

• Back Up and Restore 389 Directory Server — native backup, bak2db, and online-backup changelog implications
• dsconf commands — repl-agmt, replication get-ruv, and agreement initialization
• dsctl commands — offline db2ldif --replication, ldif2db, and bak2db

Tested on: Rocky Linux 10.2; 389 Directory Server 3.2.0.

The lab used ldap1.example.com (supplier1 on port 389; consumer1 on port 1389) and ldap2.example.com (supplier2 on port 1389). The full eight-step supplier recovery workflow, bak2db restore with reinitialization, offline db2ldif/ldif2db, and three-replica convergence polling were run end to end in that environment.

Directory Server instance names are local to each host. In this lab, consumer1 on ldap1.example.com and supplier2 on ldap2.example.com both use the local instance name ldap2.

Therefore, always run dsctl and instance-name-based dsconf commands on the host identified in the surrounding instructions. Use an LDAP or LDAPS URL when administering an explicitly remote server.

Understand Why Replica Recovery Is Different

A standalone restore replaces one server's database. A replicated restore must also account for:

Replica update vector (RUV)
Replication changelog
Replica ID
Supplier and consumer roles
Replication agreements
Changes accepted after the backup
Changelog retention
Conflict resolution

The risk in one line:

Backup at 01:00 → other suppliers accept writes until 10:00
→ failed supplier restored from 01:00 backup
→ restored data is nine hours behind the topology

The database can start while replication state is no longer suitable for incremental synchronization.

Default rule:

When a healthy current supplier exists,
prefer reinitialization over restoring an old replica backup.

Point-in-time recovery is a topology-wide decision. Stop or isolate all writers, restore one authoritative supplier to the selected recovery point, and initialize every other replica from it. Do not reconnect one historical replica to suppliers that retain newer data.

If only one supplier is restored, its consumers must be reinitialized. A consumer can resume incrementally only when the supplier still retains every required changelog update.

Record the Replication Topology Before Recovery

Document every server before you disable agreements or restore data:

Record the replicated suffix, backend name, agreement names, replica IDs, replication bind DN, TLS mode, fractional replication settings, changelog maximum age, last known successful update, and backup timestamps.

List agreements on the local supplier:

dsconf -y /root/dm.pw ldap1 repl-agmt list --suffix "dc=example,dc=com"

Sample output:

dn: cn=ldap1-to-consumer1,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
cn: ldap1-to-consumer1
nsDS5ReplicaHost: ldap1.example.com
nsDS5ReplicaPort: 1389

dn: cn=ldap1-to-supplier2,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
cn: ldap1-to-supplier2
nsDS5ReplicaHost: ldap2.example.com
nsDS5ReplicaPort: 1389

Check agreement status before you change anything:

dsconf -y /root/dm.pw ldap1 repl-agmt status     --suffix "dc=example,dc=com" ldap1-to-supplier2

Sample output:

Status For Agreement: "ldap1-to-supplier2" (ldap2.example.com:1389)
Replica Enabled: on
Update In Progress: FALSE
Last Update Status: Error (0) Replica acquired successfully: Incremental update succeeded
Last Init Status: Error (0) Total update succeeded
Replication Status: Not in Synchronization: supplier (6a5b1345000000020000) consumer (6a5b12bb000000010000) State (green) Reason (error (0) replica acquired successfully: incremental update succeeded)

State (green) does not override Not in Synchronization. Green can mean replication is functioning while the consumer is still behind. Do not consider recovery complete until lag closes and data convergence tests pass.

Display the local RUV:

dsconf -y /root/dm.pw ldap1 replication get-ruv --suffix "dc=example,dc=com"

Sample output:

RUV:        {replica 1 ldap://ldap1.example.com:389} 6a5b1207000000010000 6a5b12bb000000010000
Replica ID: 1
LDAP URL:   ldap://ldap1.example.com:389
Min CSN:    2026-07-18 05:41:27 (6a5b1207000000010000)
Max CSN:    2026-07-18 05:44:27 (6a5b12bb000000010000)

RUV:        {replica 2 ldap://ldap2.example.com:1389} 6a5b10ad000100020000 6a5b12d4000000020000
Replica ID: 2
LDAP URL:   ldap://ldap2.example.com:1389
Min CSN:    2026-07-18 05:35:41 1 0 (6a5b10ad000100020000)
Max CSN:    2026-07-18 05:44:52 (6a5b12d4000000020000)

Save agreement status, RUV output, and replication status from every surviving server before recovery.

Isolate the Failed or Restored Server

Before recovery:

1. Remove the server from the load balancer.
2. Stop application writes to that host.
3. Disable automatic service restart where necessary.
4. Block or disable every replication path touching the restored server.
5. Preserve logs and configuration immediately.
6. Record the backup timestamp and Directory Server version.
7. Confirm which remaining supplier has the newest valid data.

Disabling one outbound agreement is only an example. Repeat the action for every outbound agreement and disable the corresponding inbound agreements on peer suppliers, or isolate replication at the network layer. Enable agreements again only in the intended recovery order.

For supplier1 in this topology, block all three paths:

dsconf -y /root/dm.pw ldap1 repl-agmt disable     --suffix "dc=example,dc=com" ldap1-to-supplier2

dsconf -y /root/dm.pw ldap1 repl-agmt disable     --suffix "dc=example,dc=com" ldap1-to-consumer1

On ldap2.example.com, disable the inbound path back to the isolated server:

# Run on ldap2.example.com:
# local instance ldap2 is supplier2
dsconf -y /root/dm.pw ldap2 repl-agmt disable --suffix "dc=example,dc=com" supplier2-to-ldap1

Sample output:

Agreement has been disabled

Confirm each agreement is off before you restore or reinitialize:

dsconf -y /root/dm.pw ldap1 repl-agmt status     --suffix "dc=example,dc=com" ldap1-to-supplier2

Sample output:

Status For Agreement: "ldap1-to-supplier2" (ldap2.example.com:1389)
Replica Enabled: off
Last Update Status: Error (0) Replica acquired successfully: agreement disabled

Do not allow a restored supplier to reconnect automatically before you decide which server contains authoritative data. A current supplier initialized from an uninitialized or stale server can overwrite the current database.

Preserve logs and configuration immediately:

/var/log/dirsrv/slapd-ldap1/errors
/etc/dirsrv/slapd-ldap1/
/var/lib/dirsrv/slapd-ldap1/bak/

Preserve database files only through:

• dsconf backup create
• dsctl db2bak with the instance stopped
• A storage snapshot taken while ns-slapd is stopped or properly quiesced

Do not use an ordinary recursive copy of the live LMDB directory as a recovery backup.

Do not delete the changelog, RUV entries, or replica configuration as an initial troubleshooting action.

Choose Restore or Reinitialization

Ask:

Is another healthy supplier available?
Is that supplier more current than the backup?
Was the backup online or offline?
How old is the backup?
Were writes accepted after the backup?
Does the supplier still retain those changes in its changelog?
Is the failed server a supplier, hub, or consumer?
Did every server fail, or only one?

Do not choose restore solely because a backup exists.

Is a healthy current supplier available?
        |
        +-- Yes → reinitialize consumer, hub, or failed supplier
        |
        +-- No → restore one authoritative supplier backup,
                 validate it, then initialize the remaining topology

Recover Consumers and Hubs

A read-only consumer contains no unique writes for the replicated suffix. The consumer can still have host-specific configuration, certificates, logs, and non-replicated local data. When a healthy supplier exists, reinitialization is normally simpler than restoring an old consumer backup.

Reinitialize a consumer online

Run initialization from the supplier that owns the agreement:

dsconf -y /root/dm.pw ldap1 repl-agmt init     --suffix "dc=example,dc=com" ldap1-to-consumer1

Sample output:

Agreement initialization started...

Monitor progress on the supplier:

dsconf -y /root/dm.pw ldap1 repl-agmt init-status     --suffix "dc=example,dc=com" ldap1-to-consumer1

Sample output when initialization completes:

Agreement successfully initialized.

Check normal agreement status:

dsconf -y /root/dm.pw ldap1 repl-agmt status     --suffix "dc=example,dc=com" ldap1-to-consumer1

Sample output:

Status For Agreement: "ldap1-to-consumer1" (ldap1.example.com:1389)
Replica Enabled: on
Last Init Status: Error (0) Total update succeeded
Last Update Status: Error (0) Replica acquired successfully: Incremental update succeeded
Number Of Changes Sent: 2:1/0

Use online initialization for a manageable database and a small number of consumers.

Recover a hub

A hub is read-only but maintains a changelog because it forwards updates to downstream consumers.

Healthy supplier → reinitialize hub → validate supplier-to-hub replication
→ validate or reinitialize downstream consumers

Initialize the hub from its supplier with the same repl-agmt init and init-status commands, using the supplier-to-hub agreement name. Do not initialize consumers from the hub until supplier-to-hub initialization completes successfully.

Recover One Failed Supplier When Another Is Healthy

Example:

supplier1 <----> supplier2
supplier1 failed; supplier2 continued accepting writes

Treat supplier2 as the current authoritative source.

In the lab, supplier1 was restored from ldap1-pre-restore-lab while supplier2 held a newer marker entry. The marker was absent on the restored supplier1 until supplier2-to-ldap1 initialization completed.

Recommended workflow:

1. Isolate supplier1.
2. Enable supplier2-to-ldap1.
3. Initialize supplier1 from supplier2.
4. Validate supplier2 → supplier1 replication.
5. Enable ldap1-to-supplier2.
6. Run the bidirectional add/modify/delete test between supplier1 and supplier2 only. Downstream consumer agreements remain disabled until step 7.
7. Enable and reinitialize consumer1 when required.
8. Re-run convergence checks on every replica, then return supplier1 to client traffic.

On ldap2.example.com, enable the authoritative path to the rebuilt server before you start initialization. A total initialization should not run through a disabled agreement:

# Run on ldap2.example.com:
# local instance ldap2 is supplier2
dsconf -y /root/dm.pw ldap2 repl-agmt enable --suffix "dc=example,dc=com" supplier2-to-ldap1

Sample output:

Agreement has been enabled

Initialize through the supplier-owned agreement from supplier2 to supplier1:

# Run on ldap2.example.com:
# local instance ldap2 is supplier2
dsconf -y /root/dm.pw ldap2 repl-agmt init --suffix "dc=example,dc=com" supplier2-to-ldap1

Sample output:

Agreement initialization started...

Monitor initialization from the initiating supplier:

# Run on ldap2.example.com:
dsconf -y /root/dm.pw ldap2 repl-agmt init-status --suffix "dc=example,dc=com" supplier2-to-ldap1

Sample output:

Agreement successfully initialized.

Check the authoritative agreement status:

# Run on ldap2.example.com:
dsconf -y /root/dm.pw ldap2 repl-agmt status --suffix "dc=example,dc=com" supplier2-to-ldap1

Sample output:

Status For Agreement: "supplier2-to-ldap1" (ldap1.example.com:389)
Replica Enabled: on
Last Init Status: Error (0) Total update succeeded
Last Update Status: Error (0) Replica acquired successfully: Incremental update succeeded
Number Of Changes Sent: 2:2/0

The Error (0) prefix is normal in agreement status output; read the message after it. Avoid restoring an older supplier backup and then treating it as the source for the healthy, newer supplier.

After initialization and one-way validation from the authoritative supplier, enable the reverse agreement. Then run the complete bidirectional convergence test.

dsconf -y /root/dm.pw ldap1 repl-agmt enable --suffix "dc=example,dc=com" ldap1-to-supplier2

Sample output:

Agreement has been enabled

Confirm the reverse agreement is active:

dsconf -y /root/dm.pw ldap1 repl-agmt status --suffix "dc=example,dc=com" ldap1-to-supplier2

Sample output:

Status For Agreement: "ldap1-to-supplier2" (ldap2.example.com:1389)
Replica Enabled: on
Last Init Status: Error (0) Total update succeeded
Last Update Status: Error (0) Replica acquired successfully: Incremental update succeeded

For consumer1, do not merely enable its agreement if it became stale during recovery. Enable and reinitialize it when required:

dsconf -y /root/dm.pw ldap1 repl-agmt enable --suffix "dc=example,dc=com" ldap1-to-consumer1

Sample output:

Agreement has been enabled

dsconf -y /root/dm.pw ldap1 repl-agmt init --suffix "dc=example,dc=com" ldap1-to-consumer1

Sample output:

Agreement initialization started...

Monitor consumer initialization:

dsconf -y /root/dm.pw ldap1 repl-agmt init-status --suffix "dc=example,dc=com" ldap1-to-consumer1

Sample output:

Agreement successfully initialized.

Enable agreements in the intended recovery order: authoritative supplier to rebuilt server first, then reverse supplier paths, then downstream consumers.

Restore a Supplier When No Healthy Source Exists

Use this workflow when the failed supplier held the newest available data, no healthy supplier can initialize it, or recovery must use a native backup.

When a healthy current supplier exists, initialize the restored or rebuilt server from that supplier instead of treating this section as the primary path.

When no healthy supplier exists, restore one supplier as the recovery authority, keep every other replica offline, validate it, and initialize all other suppliers, hubs, and consumers from that restored supplier.

Verify backup checksums only when your backup workflow created a manifest at backup time. A normal Directory Server backup does not automatically create SHA256SUMS:

backup=/var/lib/dirsrv/slapd-ldap1/bak/ldap1-pre-restore-lab

test -f "$backup/SHA256SUMS" || {
    echo "SHA256SUMS was not created with this backup" >&2
    exit 1
}

cd "$backup" && sha256sum -c SHA256SUMS

Sample output:

dse_index.ldif: OK
dse_instance.ldif: OK
INFO.mdb: OK
data.mdb: OK

Checksum verification works only when the checksum manifest was created at backup time and stored separately or protected with the archive. Creating checksums after suspected corruption does not validate the original backup.

If the original instance configuration is still intact, bak2db restores the databases into that instance.

If the host or instance was rebuilt, restore or recreate the compatible instance configuration, certificates, custom schema, and encryption keys before returning the server to service. bak2db does not automatically apply files stored under config_files/. You may also need dse.ldif, the NSS certificate database, custom schema, password or PIN files, replication agreement configuration, and plug-in configuration.

Stop the instance before offline restore:

dsctl ldap1 stop

Sample output:

Instance "ldap1" has been stopped

Restore all databases from the native backup:

dsctl ldap1 bak2db "$backup"

Sample output:

bak2db successful

Start the instance while it remains isolated from client traffic and replication:

dsctl ldap1 start

Review the RUV on the recovered supplier:

dsconf -y /root/dm.pw ldap1 replication get-ruv --suffix "dc=example,dc=com"

Sample output observed in this lab after an isolated bak2db restore on 389 Directory Server 3.2.0 (before any reinitialization):

RUV:        {replica 1 ldap://ldap1.example.com:389}
Replica ID: 1
LDAP URL:   ldap://ldap1.example.com:389
Min CSN:    1970-01-01 00:00:00 (00000000000000000000)
Max CSN:    1970-01-01 00:00:00 (00000000000000000000)

That zeroed RUV was the result observed in this lab; your output may differ by backup type, package version, and whether replication metadata survived the restore. When no healthy supplier exists, treat the restored server as the recovery authority, keep every other replica offline, validate it, and initialize the remaining topology from it.

Review the error log for RUV mismatch, changelog recreation, missing CSN, reinitialization required, and agreement errors. After restoring the authoritative supplier, initialize downstream hubs, consumers, and other suppliers from this recovered server.

Understand Backup Type, Changelog Retention, and Offline Initialization

Online backup restore

An online backup is created while updates can continue. Restoring a database from an online backup clears the changelog on the restored replica. When another healthy current supplier exists, reinitialize the restored server from that supplier. When no healthy source exists, restore one server as the recovery authority, keep all other replicas offline, validate the recovered data, and initialize every other supplier, hub, and consumer from it. Incremental history that was not present in the backup cannot be recovered.

Offline backup restore

An offline backup is taken while the instance is stopped. It does not by itself guarantee incremental recovery when other suppliers continued accepting writes, the backup is older than changelog retention, or replica configuration changed after the backup.

Check configured changelog retention on the supplier:

dsconf -y /root/dm.pw ldap1 replication get-changelog --suffix "dc=example,dc=com"

Sample output:

dn: cn=changelog,cn=userroot,cn=ldbm database,cn=plugins,cn=config
nsslapd-changelogmaxage: 7d

If the supplier cannot locate the required starting CSN after a consumer restore, error logs can report:

Data required to update replica has been purged from the changelog
Changelog data is missing
Can't locate CSN in the changelog
Consumer must be reinitialized

Do not increase changelog age after required changes have been purged. A larger retention value affects future changes; it cannot recreate missing history.

Offline initialization for large replicas

Use offline initialization when the database is very large, online initialization times out, or many consumers require initialization.

The example below rebuilds consumer1 on ldap1.example.com. The local instance name is ldap2 and its LDAP URL is ldap://ldap1.example.com:1389. Do not confuse that instance with supplier2 on ldap2.example.com:1389.

Stop the authoritative supplier:

dsctl ldap1 stop

Export with replication metadata:

dsctl ldap1 db2ldif --replication userroot /var/lib/dirsrv/slapd-ldap1/ldif/replica-init.ldif

Sample output:

ldiffile: /var/lib/dirsrv/slapd-ldap1/ldif/replica-init.ldif
db2ldif successful

db2ldif requires a stopped instance. Start the supplier again after export:

dsctl ldap1 start

Sample output:

Instance "ldap1" has been started

Transfer the file securely to the consumer1 instance ldif directory and set ownership, mode, and SELinux context before import:

chown dirsrv:dirsrv /var/lib/dirsrv/slapd-ldap2/ldif/replica-init.ldif

chmod 600 /var/lib/dirsrv/slapd-ldap2/ldif/replica-init.ldif

restorecon -v /var/lib/dirsrv/slapd-ldap2/ldif/replica-init.ldif

Stop the consumer1 instance:

# Run on ldap1.example.com:
# local instance ldap2 is consumer1
dsctl ldap2 stop

Sample output:

Instance "ldap2" has been stopped

Import the replication-aware LDIF on consumer1:

# Run on ldap1.example.com:
dsctl ldap2 ldif2db userroot /var/lib/dirsrv/slapd-ldap2/ldif/replica-init.ldif

Sample output when the import is clean:

ldif2db successful

Start consumer1:

# Run on ldap1.example.com:
dsctl ldap2 start

Sample output:

Instance "ldap2" has been started

After manual offline import, verify with repl-agmt status, actual replicated add/modify/delete operations, RUV inspection, and ds-replcheck. Do not rely on repl-agmt init-status alone; it reports agreement initialization-task status and may reflect an earlier online initialization rather than prove the manually imported database is correct.

Compare the supplier to the instance you imported. Here -m is the supplier and -r is consumer1 (ldap://ldap1.example.com:1389):

ds-replcheck state -D "cn=Directory Manager" -W -m ldap://ldap1.example.com:389 -r \
ldap://ldap1.example.com:1389 -b "dc=example,dc=com"

Sample output:

Replication State: Supplier and Replica are in perfect synchronization

For a deeper comparison:

ds-replcheck online -D "cn=Directory Manager" -W -m ldap://ldap1.example.com:389 -r \
ldap://ldap1.example.com:1389 -b "dc=example,dc=com"

Do not use ordinary LDIF export as a substitute for replication-aware initialization.

Recover the Entire Replication Topology

Use this when every supplier, hub, and consumer is unavailable or unusable.

1. Choose the backup with the newest valid recovery point, verified checksums, compatible Directory Server version, and known replica role.
2. Restore or import the first authoritative supplier while all other replicas remain offline.
3. Validate directory data, binds, ACIs, schema, indexes, TLS, plug-ins, RUV, and database health.
4. Initialize the second supplier from the first, then verify the reverse agreement.
5. Initialize hubs and consumers one layer at a time.
6. Return client traffic gradually: one read-only consumer, one supplier for controlled writes, then the remaining servers.

Do not merge unrelated native backups from servers taken at different times.

Validate the RUV and Replication Before Returning to Service

Display the RUV on each replica:

dsconf -y /root/dm.pw ldap1 replication get-ruv --suffix "dc=example,dc=com"

dsconf -y /root/dm.pw ldap2 replication get-ruv --suffix "dc=example,dc=com"

Review replica ID, supplier URL, minimum CSN, maximum CSN, and obsolete supplier entries. Every writable supplier must have a unique replica ID. Do not reuse an active supplier's replica ID for a replacement server.

RUV output does not need to be textually identical at every instant. Check unique supplier IDs, compatible replica generation, expected supplier entries, and closing CSN lag. Use ds-replcheck and data tests to establish convergence.

Check every agreement:

dsconf -y /root/dm.pw ldap1 repl-agmt status --suffix "dc=example,dc=com" ldap1-to-supplier2

Run a complete bidirectional convergence test before you return servers to traffic. During supplier recovery, while consumer agreements are still disabled, poll only the writable suppliers. After you reinitialize downstream consumers, include every consumer URI in the same poll loops.

Replication is asynchronous. Poll each replica until the expected value appears or until a defined timeout expires. Do not assume that the change will be visible immediately after ldapadd, ldapmodify, or ldapdelete returns.

Create the entry on supplier1:

New entries in this section are added with the ldapadd command.

ldapadd -x -H ldap://ldap1.example.com:389 -D "cn=Directory Manager" -y /root/dm.pw <<'EOF'
dn: cn=e2e-restore-test,ou=People,dc=example,dc=com
objectClass: top
objectClass: person
cn: e2e-restore-test
sn: restore-test
description: created on supplier1
EOF

Sample output:

adding new entry "cn=e2e-restore-test,ou=People,dc=example,dc=com"

Poll every replica until the entry appears:

for URI in ldap://ldap1.example.com:389 ldap://ldap2.example.com:1389 ldap://ldap1.example.com:1389; do
  MATCHED=0
  for attempt in $(seq 1 30); do
    VALUE=$(ldapsearch -LLL -x -H "$URI" -D "cn=Directory Manager" -y /root/dm.pw -b "cn=e2e-restore-test,ou=People,dc=example,dc=com" -s base description 2>/dev/null | awk -F': ' '/^description:/ {print $2}')
    if [ "$VALUE" = "created on supplier1" ]; then
      printf '%s converged\n' "$URI"
      MATCHED=1
      break
    fi
    sleep 2
  done
  if [ "$MATCHED" -ne 1 ]; then
    printf '%s did not converge before timeout\n' "$URI" >&2
  fi
done

Sample output:

ldap://ldap1.example.com:389 converged
ldap://ldap2.example.com:1389 converged
ldap://ldap1.example.com:1389 converged

Modify the entry on supplier2:

Entry updates in this section use the ldapmodify command.

ldapmodify -x -H ldap://ldap2.example.com:1389 -D "cn=Directory Manager" -y /root/dm.pw <<'EOF'
dn: cn=e2e-restore-test,ou=People,dc=example,dc=com
changetype: modify
replace: description
description: modified on supplier2
EOF

Poll every replica until the modified value appears:

for URI in ldap://ldap1.example.com:389 ldap://ldap2.example.com:1389 ldap://ldap1.example.com:1389; do
  MATCHED=0
  for attempt in $(seq 1 30); do
    VALUE=$(ldapsearch -LLL -x -H "$URI" -D "cn=Directory Manager" -y /root/dm.pw -b "cn=e2e-restore-test,ou=People,dc=example,dc=com" -s base description 2>/dev/null | awk -F': ' '/^description:/ {print $2}')
    if [ "$VALUE" = "modified on supplier2" ]; then
      printf '%s converged\n' "$URI"
      MATCHED=1
      break
    fi
    sleep 2
  done
  if [ "$MATCHED" -ne 1 ]; then
    printf '%s did not converge before timeout\n' "$URI" >&2
  fi
done

Sample output:

ldap://ldap1.example.com:389 converged
ldap://ldap2.example.com:1389 converged
ldap://ldap1.example.com:1389 converged

Delete the entry from supplier1:

Removals in this section use the ldapdelete command.

ldapdelete -x -H ldap://ldap1.example.com:389 -D "cn=Directory Manager" -y /root/dm.pw \
"cn=e2e-restore-test,ou=People,dc=example,dc=com"

Sample output:

deleting entry "cn=e2e-restore-test,ou=People,dc=example,dc=com"

Poll every replica until the entry count reaches zero:

for URI in ldap://ldap1.example.com:389 ldap://ldap2.example.com:1389 ldap://ldap1.example.com:1389; do
  MATCHED=0
  for attempt in $(seq 1 30); do
    COUNT=$(ldapsearch -LLL -x -H "$URI" -D "cn=Directory Manager" -y /root/dm.pw -b "ou=People,dc=example,dc=com" "(&(cn=e2e-restore-test)(sn=restore-test))" dn 2>/dev/null | grep -c '^dn:')
    if [ "$COUNT" -eq 0 ]; then
      printf '%s remaining=0\n' "$URI"
      MATCHED=1
      break
    fi
    sleep 2
  done
  if [ "$MATCHED" -ne 1 ]; then
    printf '%s did not converge before timeout\n' "$URI" >&2
  fi
done

Sample output when convergence is complete:

ldap://ldap1.example.com:389 remaining=0
ldap://ldap2.example.com:1389 remaining=0
ldap://ldap1.example.com:1389 remaining=0

ds-replcheck state reports replication lag between two servers. Use ds-replcheck online for a deeper entry-by-entry comparison:

ds-replcheck state -D "cn=Directory Manager" -W -m ldap://ldap1.example.com:389 -r \
ldap://ldap2.example.com:1389 -b "dc=example,dc=com"

Sample output:

Replication State: Supplier and Replica are in perfect synchronization

Also compare representative entry counts, password binds, large groups, memberOf, custom schema entries, and fractional replication exclusions.

Do not rely only on agreement status text. Verify actual bidirectional data convergence.

List replication conflicts after recovery:

dsconf -y /root/dm.pw ldap1 repl-conflict list "dc=example,dc=com"

Sample output when no conflicts exist:

There were no conflict entries found under the suffix

Resolve conflicts only after determining which entry should be retained. Do not automatically delete every conflict or glue entry immediately after recovery.

Troubleshoot Replica Recovery

Obsolete RUV metadata and cleanallruv

A restored RUV may contain metadata for a permanently decommissioned supplier. Use cleanallruv only when that supplier will never rejoin, all valid updates from it have been replicated, the replica ID is not in use, every active supplier is reachable, and a current backup exists. Do not run cleanallruv merely because two RUV outputs differ.

Build a Replica Recovery Runbook

Record:

Topology diagram
Supplier and consumer roles
Replica IDs
Agreement names
Replication bind credentials location
Backup locations and timestamps
Changelog retention
Initialization commands
Recovery order
RUV outputs
Validation searches
Conflict-resolution owner
Load-balancer removal and return steps

Test periodically in an isolated environment:

1. Rebuild one consumer.
2. Rebuild one supplier from another supplier.
3. Restore the only supplier from backup.
4. Recover a hub and its consumers.
5. Restore the entire topology.

Measure time to restore the first authoritative supplier, time to initialize each replica, time to validate data, time to restore application traffic, and actual RPO and RTO.

Recovery recommendations

• Prefer reinitialization when a healthy current supplier exists.
• Restore a backup only after identifying the authoritative data source.
• Isolate a restored supplier before enabling replication.
• Never initialize a current supplier from a stale restored replica.
• Use online initialization for smaller datasets and offline initialization for large databases.
• Compare backup age with changelog retention.
• Expect downstream reinitialization after restoring an online supplier backup.
• Restore one authoritative supplier first during topology-wide recovery.
• Initialize the remaining topology one layer at a time.
• Compare RUV and replica IDs before reconnecting suppliers.
• Test actual replicated writes, not only agreement status.
• Keep every recovered server out of the load balancer until validation is complete.

What's next

After you complete this guide, continue with:

• Migrate 389 Directory Server to a New Server — planned host replacement and cutover
• Update and Upgrade 389 Directory Server Safely — package updates after replica recovery
• Migrate 389 Directory Server from Berkeley DB to LMDB — backend conversion in replicated topologies
• Configure Low Disk Space Protection in 389 Directory Server — disk pressure during reinitialization

Summary

Replica recovery is a replication-state decision, not only a database restore. Record topology and RUVs, isolate every replication path touching the failed server, identify the newest authoritative replica, and prefer reinitialization when a healthy current supplier exists. Restore a native backup only when no newer source is available or when you are executing a topology-wide point-in-time recovery. Validate with agreement status, RUV inspection, ds-replcheck, and complete add/search/modify/delete tests before returning servers to application traffic.

References

• 389 Directory Server documentation
• Red Hat Directory Server 13 — configuring and managing replication
• Red Hat Directory Server 13 — backing up and restoring Directory Server
• Red Hat Directory Server 13 — comparing two Directory Server instances
• Red Hat Directory Server 13 — exporting and importing data
• Red Hat Directory Server 13 — monitoring server and database activity