Monitor 389 Directory Server Replication and Lag with dsconf

Monitor 389 Directory Server replication with dsconf: check agreement status, topology, CSNs and lag, analyze access logs, and configure alerts.

Published

Updated

Read time 18 min read

Reviewed byDeepak Prasad

Monitor 389 Directory Server replication status lag and topology with dsconf replication monitor

Replication can look healthy from one supplier while another path is stalled, credentials expired, or lag is climbing quietly. This guide shows how to monitor 389 Directory Server replication from the command line: pick the right dsconf subcommand, read status fields and CSNs, generate topology and historical lag reports, and wire basic alerts.

For agreement creation, schedules, and forced updates, see manage replication agreements. For changelog retention that affects catch-up windows, see manage the replication changelog. When replication fails or falls out of sync, see troubleshoot replication failures. For a deeper entry-level comparison between two hosts, use ds-replcheck as shown below.

Tested on: Rocky Linux 10.2; 389 Directory Server 3.2.0.


Choose the right replication monitoring command

Start with the command that matches your question. You do not need every tool on every check.

Command Best use
repl-agmt status Detailed status of one agreement
replication status Status of all agreements on one local instance
replication monitor Live report across the discovered topology
replication lag-report Historical CSN-based lag analysis from access logs
dsctl healthcheck --check replication Automated replication health checks
ds-replcheck state Compare replication and RUV state between two URLs
ds-replcheck online Entry and attribute comparison between two URLs

repl-agmt status and replication status read live replica state. replication lag-report looks backward through access logs. Keep those roles separate when you design monitoring.


Prepare the replication monitoring lab

Examples use the shared replication lab from manage replication agreements. Commands run on ldap1.example.com against instance ldap1 unless noted.

Setting Value
Source supplier ldap1.example.com (instance ldap1, port 389)
Primary destination consumer1.example.com (instance ldap2, port 1389)
Primary agreement ldap1-to-consumer1
Secondary agreement (unhealthy example) ldap1-to-supplier2supplier2.example.com:1636
Replicated suffix dc=example,dc=com

Password files under /root with mode 600:

File Purpose
/root/dm.pw Directory Manager on ldap1
/root/consumer1-dm.pw Directory Manager on consumer1 (ldap2 instance)
/root/replcheck.pw Password for the bind DN used by ds-replcheck on both instances (mode 600)

Both ldap1 and ldap2 run on the same Rocky Linux lab host, so later verification commands can reach ldap2 through ldap://127.0.0.1:1389. In a multi-host deployment, replace the loopback URL with ldap://consumer1.example.com:1389.

Confirm consumer1.example.com resolves through DNS or /etc/hosts on every host that runs these commands. If you already completed single-supplier replication and ldap1-to-consumer1 exists, start at Check replication status with dsconf.

Before you trust status output or build alerts, confirm the basics:

Every consumer in the path must be initialized. An uninitialized consumer can show agreements and even partial status while data is missing. See initialize and reinitialize replicas if Last Init Status is not Total update succeeded.

Verify network access from each supplier to every agreement hostname and port. A firewall change or stale DNS entry often appears first as Replica Status: Unreachable or Replication Lag Time: unavailable.

Synchronize system clocks on every replica host. CSNs embed time, lag calculations compare timestamps, and skew can make healthy replication look out of sync. On RHEL-family systems, confirm chronyd is active:

bash
systemctl is-active chronyd

Sample output:

output
active

List every replicated suffix and outgoing agreement on the instance you monitor:

bash
dsconf -y /root/dm.pw ldap1 repl-agmt list --suffix "dc=example,dc=com"

Sample output (trimmed):

output
cn: ldap1-to-consumer1
cn: ldap1-to-supplier2

The list tells you which agreement names to pass to repl-agmt status and which destinations need credentials in ~/.dsrc.

Confirm monitoring credentials can bind to remote instances. Topology-wide replication monitor follows agreements and may need Directory Manager or another read-capable identity on each host it visits.

Enable and retain access logs when you plan to run replication lag-report. The analyzer reads CSN markers from access logs on every replica in scope. Short retention or disabled access logging produces incomplete historical reports.


Check replication status with dsconf

Check one agreement

Use repl-agmt status when you already know the agreement name and want the full runtime picture for that single link. ldap1-to-consumer1 is the primary lab agreement from ldap1 to consumer1.example.com:1389.

Pass consumer credentials when supplier and destination Directory Manager passwords differ. Without them, synchronization and lag fields may be incomplete even when replication works.

bash
dsconf -y /root/dm.pw ldap1 repl-agmt status --suffix "dc=example,dc=com" --bind-dn "cn=Directory Manager" --bind-passwd-file /root/consumer1-dm.pw ldap1-to-consumer1

Sample output from a synchronized agreement:

output
Status For Agreement: "ldap1-to-consumer1" (consumer1.example.com:1389)
Replica Enabled: on
Update In Progress: FALSE
Last Update Start: 20260720051158Z
Last Update End: 20260720051158Z
Number Of Changes Sent: 1:5/0 
Number Of Changes Skipped: None
Last Update Status: Error (0) Replica acquired successfully: Incremental update succeeded
Last Init Start: 20260720021937Z
Last Init End: 20260720021940Z
Last Init Status: Error (0) Total update succeeded
Reap Active: 0
Replication Status: In Synchronization
Replication Lag Time: 00:00:00

Replication Status: In Synchronization with 00:00:00 lag is the steady state you want. In these lines, the accompanying messages — Incremental update succeeded and Total update succeeded — confirm success. Do not treat Error (0) alone as a health signal.

repl-agmt init-status answers whether a total update task is still running after repl-agmt init:

bash
dsconf -y /root/dm.pw ldap1 repl-agmt init-status --suffix "dc=example,dc=com" ldap1-to-consumer1

Sample output:

output
Agreement successfully initialized.

The same instance can hold another agreement that is not in sync. ldap1-to-supplier2 points at a remote host that was unreachable during this test:

bash
dsconf -y /root/dm.pw ldap1 repl-agmt status --suffix "dc=example,dc=com" --bind-dn "cn=Directory Manager" --bind-passwd-file /root/dm.pw ldap1-to-supplier2

Sample output (trimmed):

output
Replication Status: Not in Synchronization: supplier (6a5dae1e000000010000) consumer (Unavailable) State (green) Reason (error (0) no replication sessions started since server startup)
Replication Lag Time: unavailable

Do not treat State (green) as an overall health result. In this output, consumer (Unavailable), Not in Synchronization, and unavailable lag clearly show that the path is unhealthy.

Check every agreement on the server

When you only need a summary of outgoing agreements on the local instance, replication status is faster than calling repl-agmt status for each name.

bash
dsconf -y /root/dm.pw ldap1 replication status --suffix "dc=example,dc=com" --bind-dn "cn=Directory Manager" --bind-passwd-file /root/consumer1-dm.pw

The default output prints one Python dictionary-style record per agreement. This is readable interactively but is not valid JSON because it uses single quotes. This lab returned two lines:

output
{'agmt-name': ['ldap1-to-consumer1'], 'replica': ['consumer1.example.com:1389'], 'replica-enabled': ['on'], 'update-in-progress': ['FALSE'], 'last-update-start': ['20260720051158Z'], 'last-update-end': ['20260720051158Z'], 'number-changes-sent': ['1:5/0 '], 'number-changes-skipped': ['unavailable'], 'last-update-status': ['Error (0) Replica acquired successfully: Incremental update succeeded'], 'last-init-start': ['20260720021937Z'], 'last-init-end': ['20260720021940Z'], 'last-init-status': ['Error (0) Total update succeeded'], 'reap-active': ['0'], 'replication-status': ['In Synchronization'], 'replication-lag-time': ['00:00:00']}
{'agmt-name': ['ldap1-to-supplier2'], 'replica': ['supplier2.example.com:1636'], 'replica-enabled': ['on'], 'update-in-progress': ['FALSE'], 'last-update-start': ['19700101000000Z'], 'last-update-end': ['19700101000000Z'], 'number-changes-sent': ['0'], 'number-changes-skipped': ['unavailable'], 'last-update-status': ['Error (0) No replication sessions started since server startup'], 'last-init-start': ['20260719100536Z'], 'last-init-end': ['20260719100549Z'], 'last-init-status': ['Error (0) Total update succeeded'], 'reap-active': ['0'], 'replication-status': ['Not in Synchronization: supplier (6a5dae1e000000010000) consumer (Unavailable) State (green) Reason (error (0) no replication sessions started since server startup)'], 'replication-lag-time': ['unavailable']}

The first line is the healthy ldap1-to-consumer1 path. The second shows ldap1-to-supplier2 out of sync with replication-lag-time set to unavailable.

Add the global -j option when a script needs structured JSON:

bash
dsconf -j -y /root/dm.pw ldap1 replication status --suffix "dc=example,dc=com" --bind-dn "cn=Directory Manager" --bind-passwd-file /root/consumer1-dm.pw

replication status applies the same --bind-dn and destination password to every agreement in the command. If remote replicas use different credentials, some synchronization fields can appear unavailable even when those replicas are reachable. Use separate repl-agmt status commands or topology monitoring with per-host .dsrc credentials in that situation. In this lab, consumer1 and supplier2 use different password files.

replication status does not walk the full topology. It reports only agreements configured on the instance you query. Use replication monitor when you need supplier, hub, and consumer sections discovered from the starting host.


Generate a complete replication topology report

dsconf replication monitor starts on one instance, follows outgoing agreements, and builds a multi-server report. It is the right tool when you need to see every role in the path from a single command.

Configure credentials first (next section) or pass -c connection strings on the command line. Add the [repl-monitor-connections] and [repl-monitor-aliases] blocks from the next section to ~/.dsrc before you run the command below without prompts.

bash
dsconf -y /root/dm.pw ldap1 replication monitor

Sample output (trimmed):

output
Supplier: Supplier-1 (ldap1.example.com:389)
--------------------------------------------
Replica Root: dc=example,dc=com
Replica ID: 1
Replica Status: Online
Max CSN: 6a5dae1e000000010000

Status For Agreement: "ldap1-to-consumer1" (consumer1.example.com:1389)
Replica Enabled: on
Update In Progress: FALSE
Last Update Start: 20260720051158Z
Last Update End: 20260720051158Z
Number Of Changes Sent: 1:5/0 
Number Of Changes Skipped: None
Last Update Status: Error (0) Replica acquired successfully: Incremental update succeeded
Replication Status: In Synchronization
Replication Lag Time: 00:00:00

Status For Agreement: "ldap1-to-supplier2" (supplier2.example.com:1636)
Replica Enabled: on
Replication Status: Not in Synchronization: supplier (6a5dae1e000000010000) consumer (Unavailable) State (green) Reason (error (0) no replication sessions started since server startup)
Replication Lag Time: unavailable

Supplier: Consumer-1 (consumer1.example.com:1389)
-------------------------------------------------
Replica Root: dc=example,dc=com
Replica ID: 65535
Replica Status: Online
Max CSN: 00000000000000000000

Supplier: Supplier-2 (supplier2.example.com:1636)
-------------------------------------------------
Replica Status: Unreachable - Can't contact LDAP server

The monitor discovered ldap1 as a supplier, followed ldap1-to-consumer1 to the consumer, and attempted ldap1-to-supplier2. Supplier-2 is unreachable in this lab, which is a realistic example of an incomplete topology report.

Place the global -j flag before the instance name so scripts receive structured data:

bash
dsconf -j -y /root/dm.pw ldap1 replication monitor

Sample output (trimmed):

output
{
    "type": "list",
    "items": [
        {
            "name": "Supplier-1 (ldap1.example.com:389)",
            "data": [
                {
                    "replica_id": "1",
                    "replica_root": "dc=example,dc=com",
                    "replica_status": "Online",
                    "maxcsn": "6a5dae1e000000010000",
                    "agmts_status": [
                        {
                            "agmt-name": ["ldap1-to-consumer1"],
                            "replication-status": ["In Synchronization"],
                            "replication-lag-time": ["00:00:00"]
                        }
                    ]
                }
            ]
        }
    ]
}

Parse replication-status and replication-lag-time from each agmts_status entry instead of scraping the human-readable report.


Configure monitoring credentials and aliases in .dsrc

Topology monitoring needs bind credentials for every host the tool visits. dsconf checks sources in this order: CLI -c / -a arguments, then ~/.dsrc, then interactive prompts.

Add two sections to ~/.dsrc:

ini
[repl-monitor-connections]
consumer1 = consumer1.example.com:1389:cn=Directory Manager:[/root/consumer1-dm.pw]
supplier2 = supplier2.example.com:1636:cn=Directory Manager:[/root/dm.pw]

[repl-monitor-aliases]
Supplier-1 = ldap1.example.com:389
Consumer-1 = consumer1.example.com:1389
Supplier-2 = supplier2.example.com:1636

Connection values use host:port:binddn:bindpw. Keys under [repl-monitor-connections] are arbitrary labels; only the values matter. Supported password forms:

  • * prompts interactively at runtime
  • [~/password-file] reads a single-line password from a protected file
  • a literal password works but is discouraged

Restrict permissions on ~/.dsrc and every password file (chmod 600). Scheduled jobs must run as the user whose home directory contains the configuration.

Aliases add a friendly label before the original host:port, making large topology reports easier to read. You can also pass aliases on the command line:

bash
dsconf -y /root/dm.pw ldap1 replication monitor \
  -a "Supplier-1=ldap1.example.com:389" \
     "Consumer-1=consumer1.example.com:1389"

CLI -c and -a values override ~/.dsrc when both are present.


Interpret replication status and lag fields

These fields appear in repl-agmt status, replication status, and replication monitor output.

Field What it tells you
Replica Enabled Agreement enabled (on) or administratively disabled (off)
Replica Status Whether the local or remote replica is Online, Unreachable, or otherwise unavailable
Update In Progress TRUE while a replication session is active; usually brief
Last Update Start / End Timestamps for the most recent incremental session
Last Update Status Result of the last incremental update; read the full message, not just Error (0)
Last Init Status Result of the last total update (repl-agmt init)
Number Of Changes Sent Activity counter since startup; can reset when the supplier restarts
Number Of Changes Skipped Changes not sent, often None or unavailable
Max CSN Highest change sequence number on that replica for the suffix
Replication Status In Synchronization or a Not in Synchronization explanation with CSNs
Replication Lag Time Time difference between supplier and consumer CSNs; 00:00:00 when caught up
Reap Active Whether a replica reaping task is running (0 = no)

Important interpretation notes:

  • Error (0) means Directory Server recorded error code zero. Read the complete message — Incremental update succeeded is healthy; No replication sessions started since server startup is not.
  • In Synchronization and zero lag are the expected steady state when changes are flowing.
  • Unavailable on lag or consumer CSN usually means connectivity, authentication, or missing consumer status data.
  • Zero lag indicates CSN-based catch-up for the monitored path; it does not establish entry-level identity. Use ds-replcheck online for a deeper comparison, preferably during a quiet period, and ignore attributes intentionally excluded by fractional replication.
  • Sent and skipped counters are activity indicators, not long-term audit totals.
  • A long interval since the last update is normal when the directory is idle and no writes occurred.

Understand CSNs without overinterpreting them

A Change Sequence Number identifies and orders a change using a timestamp, sequence number, originating replica ID, and subsequence number. For example, 6a5dae1e000000010000 can be separated as 6a5dae1e | 0000 | 0001 | 0000. The CSN format is timestamp → sequence → replica ID → subsequence.

Max CSN on a supplier shows how far that replica has advanced. Inspect the Replication Update Vector (RUV) for every replica ID participating in the suffix:

bash
dsconf -y /root/dm.pw ldap1 replication get-ruv --suffix "dc=example,dc=com"

Sample output (trimmed):

output
RUV:        {replica 1 ldap://ldap1.example.com:389} 6a5d05cc000000010000 6a5daedc000000010000
Replica ID: 1
LDAP URL:   ldap://ldap1.example.com:389
Min CSN:    2026-07-19 17:13:48 (6a5d05cc000000010000)
Max CSN:    2026-07-20 05:15:08 (6a5daedc000000010000)

Use CSNs to understand progress along a specific replication path — for example comparing supplier and consumer values in Replication Status. Do not treat lexical CSN comparison across unrelated suppliers as a complete consistency test. For stale RUV cleanup after permanent replica removal, see CleanAllRUV.

ds-replcheck uses one bind DN and password for both URLs, so the credential must be accepted by both instances. The commands below assume that cn=Directory Manager uses the same password on both lab instances and that the password is stored in /root/replcheck.pw. If the Directory Manager passwords differ, replace -D "cn=Directory Manager" in both commands with the DN of a common read-capable account and store that account's password in /root/replcheck.pw.

ds-replcheck state compares general replication and RUV state:

bash
ds-replcheck state -D "cn=Directory Manager" -y /root/replcheck.pw -m ldap://127.0.0.1:389 -r ldap://127.0.0.1:1389 -b "dc=example,dc=com"

Sample output:

output
Replication State: Supplier and Replica are in perfect synchronization

ds-replcheck online performs the entry-and-attribute comparison. Run it during a quiet window when possible; active writes can create transient differences. Pass -i to ignore attributes excluded by fractional replication:

bash
ds-replcheck online -D "cn=Directory Manager" -y /root/replcheck.pw -m ldap://127.0.0.1:389 -r ldap://127.0.0.1:1389 -b "dc=example,dc=com"

Sample output (trimmed):

output
Replication State: Supplier and Replica are in perfect synchronization

Entry Counts
=====================================================
Supplier: 788
Replica: 799

Result
=====================================================
There are replication differences between Supplier and Replica

This is not a clean comparison: the replica reports 11 more total entries than the supplier. These counts include tombstones, so the difference does not automatically mean that 11 live entries are missing. Review the untrimmed Tombstones, Conflict Entries, Missing Entries, and Entry Inconsistencies sections to identify the actual discrepancy.

state can report RUV synchronization while online still finds missing entries or attribute drift. That is why monitoring uses CSN lag, RUV state, and entry-level comparison together rather than treating any single result as definitive.


Generate historical replication lag reports

replication lag-report is separate from live monitoring. It correlates CSN markers in access logs from multiple servers, calculates global and hop-by-hop lag, and can flag changes that never reached every replica.

IMPORTANT
The CLI lag analyzer was introduced in 389 Directory Server 3.1.3 and improved in 3.2.0. Confirm availability on your host before you add it to automation.

Verify the subcommand on your instance:

bash
dsconf ldap1 replication lag-report --help

Sample output (trimmed):

output
usage: dsconf [-v] [-j] instance replication lag-report [-h] --log-dirs
                                                        LOG_DIRS
                                                        [LOG_DIRS ...]
                                                        --suffixes SUFFIXES
                                                        [SUFFIXES ...]
                                                        --output-dir
                                                        OUTPUT_DIR

lag-report reads local filesystem paths; it does not retrieve logs from remote servers. For a multi-host topology, copy or mount each replica's access logs on the analysis host and keep every server's logs in a separate directory. Each --log-dirs argument represents one server.

Basic lag report

Point the analyzer at access-log directories for every replica in scope, choose the suffix, and write reports to an output directory. --start-time and --end-time must use YYYY-MM-DD HH:MM:SS with a space between date and time — not ISO-8601 T separators. Set the window to the calendar day your rotated access logs cover. The example below uses UTC midnight through end of day; add --utc-offset "+0530" when your window represents an IST calendar day.

bash
mkdir -p /tmp/repl-lag-report

mkdir -p prints no output on success, whether it creates the directory or the directory already exists.

bash
dsconf -y /root/dm.pw ldap1 replication lag-report --log-dirs /var/log/dirsrv/slapd-ldap1 /var/log/dirsrv/slapd-ldap2 --suffixes "dc=example,dc=com" --output-dir /tmp/repl-lag-report --output-format json --start-time "2026-07-20 00:00:00" --end-time "2026-07-20 23:59:59"

Using 2026-07-20T00:00:00 instead fails immediately:

output
Error: Invalid time format. Use YYYY-MM-DD HH:MM:SS: time data '2026-07-20T00:00:00' does not match format '%Y-%m-%d %H:%M:%S'

On a successful run you see:

output
Initializing replication log analysis...
Analyzing replication logs...
Generating analysis reports...
Creating reports in formats: ['json']
Generated report files:
  summary: /tmp/repl-lag-report/replication_analysis_summary.json
  json: /tmp/repl-lag-report/replication_analysis.json

Inspect the summary:

bash
cat /tmp/repl-lag-report/replication_analysis_summary.json

Sample output (trimmed):

output
{
    "analysis_summary": {
        "total_servers": 2,
        "analyzed_logs": 32,
        "total_updates": 48,
        "minimum_lag": 0.0,
        "maximum_lag": 4.412407159805298,
        "average_lag": 0.3404512032866478,
        "updates_by_suffix": {
            "dc=example,dc=com": 48
        },
        "time_range": {
            "start": "2026-07-20 00:16:15.300149+00:00",
            "end": "2026-07-20 05:14:16.019851+00:00"
        }
    }
}

The time_range block shows which events the analyzer actually found inside your requested window.

The analyzer may log DN normalization warnings for malformed entries in older access logs. Those lines do not always stop report generation; review skipped lines when counts look too low.

Filter by lag threshold

Add --lag-time-lowest to focus on slower events (value is in seconds). Create the output directory first — mkdir -p prints no output on success:

bash
mkdir -p /tmp/repl-lag-threshold && dsconf -y /root/dm.pw ldap1 replication lag-report --log-dirs /var/log/dirsrv/slapd-ldap1 /var/log/dirsrv/slapd-ldap2 --suffixes "dc=example,dc=com" --output-dir /tmp/repl-lag-threshold --output-format csv --start-time "2026-07-20 00:00:00" --end-time "2026-07-20 23:59:59" --lag-time-lowest 1

Sample CSV header and first row:

output
Timestamp,Server,CSN,Suffix,Target DN,Global Lag (s),Duration (s),Operation Etime
2026-07-20 02:00:38,slapd-ldap1,6a5d8146000000010000,"dc=example,dc=com","uid=frac-test,ou=People,dc=example,dc=com",4.412,0.009,0.009431434

Optional filters include --only-not-replicated, --etime-lowest, and --precision (fast, balanced, full). JSON and CSV reports are available with the base package. HTML and PNG output requires python3-lib389-repl-reports.


Validate monitoring with a controlled update

A temporary LDAP modify confirms that both replication and your monitoring workflow react to real change.

Record the current agreement status:

bash
dsconf -y /root/dm.pw ldap1 repl-agmt status --suffix "dc=example,dc=com" --bind-dn "cn=Directory Manager" --bind-passwd-file /root/consumer1-dm.pw ldap1-to-consumer1

Sample output before the write (trimmed):

output
Number Of Changes Sent: 1:9/0 
Last Update Start: 20260720053142Z
Replication Lag Time: 00:00:00

Note Number Of Changes Sent and Replication Lag Time before you modify an entry.

Add a temporary description value on the supplier. Using add preserves any existing description values on the entry:

bash
ldapmodify -x -H ldap://127.0.0.1:389 -D "cn=Directory Manager" -y /root/dm.pw <<'EOF'
dn: uid=user1,ou=People,dc=example,dc=com
changetype: modify
add: description
description: monitor-test-1784524454
EOF

Sample output:

output
modifying entry "uid=user1,ou=People,dc=example,dc=com"

Re-check agreement status after a few seconds:

bash
dsconf -y /root/dm.pw ldap1 repl-agmt status --suffix "dc=example,dc=com" --bind-dn "cn=Directory Manager" --bind-passwd-file /root/consumer1-dm.pw ldap1-to-consumer1

Sample output after replication (trimmed):

output
Last Update Start: 20260720053147Z
Last Update End: 20260720053147Z
Number Of Changes Sent: 1:10/0 
Last Update Status: Error (0) Replica acquired successfully: Incremental update succeeded
Replication Lag Time: 00:00:00

The sent counter advanced from 1:9/0 to 1:10/0, the update timestamps moved forward, and lag stayed at zero.

Confirm the value on the consumer. In this lab both instances share one host, so the consumer is reached at 127.0.0.1:1389:

bash
ldapsearch -LLL -x -H ldap://127.0.0.1:1389 -D "cn=Directory Manager" -y /root/consumer1-dm.pw -b "uid=user1,ou=People,dc=example,dc=com" -s base description

Sample output:

output
dn: uid=user1,ou=people,dc=example,dc=com
description: existing-description
description: monitor-test-1784524454

Remove only the test value when you finish:

bash
ldapmodify -x -H ldap://127.0.0.1:389 -D "cn=Directory Manager" -y /root/dm.pw <<'EOF'
dn: uid=user1,ou=People,dc=example,dc=com
changetype: modify
delete: description
description: monitor-test-1784524454
EOF

Sample output:

output
modifying entry "uid=user1,ou=People,dc=example,dc=com"

Cleanup is another replicated modification, so Number Of Changes Sent and Max CSN can advance again after you remove the test value.

Run replication monitor again if you want to see Max CSN advance on the supplier section.


Add basic replication alerts

Schedule lightweight checks from cron, a systemd timer, or your monitoring platform. Prefer JSON output (dsconf -j, dsctl -j) so scripts do not parse free text.

dsctl healthcheck must run locally because it reads instance files and other local resources. Run the check separately on every replica host; it is not a remote topology-wide healthcheck.

Example health check with dsctl:

bash
dsctl -j ldap1 healthcheck --check replication

Sample output when no issues are found:

output
[]

An empty JSON array means the built-in replication checks passed. Wire a non-empty report or a failed dsctl run into your alerting pipeline. The example below requires jq:

bash
hc_rc=0
hc_json=$(dsctl -j ldap1 healthcheck --check replication) || hc_rc=$?
if [ "$hc_rc" -ne 0 ] || ! printf '%s' "$hc_json" | jq -e 'type == "array" and length == 0' >/dev/null; then
    logger -p daemon.err -t ds-repl-health \
      "ldap1 replication healthcheck reported findings"
fi

The script captures dsctl output and exit status separately, then validates that the result is an empty JSON array. A non-empty report, a non-array response, invalid JSON, or a failed dsctl invocation writes an error event to the system log. logger only records that event — it does not notify an administrator by itself. A monitoring agent, journald/rsyslog rule, or alert platform must consume the log entry.

Alert when you see:

  • Remote replica Unreachable or Unavailable
  • Invalid monitoring credentials
  • Agreement unexpectedly disabled (replica-enabled off)
  • Missing agreements, not only disabled agreements
  • replication-status containing Not in Synchronization
  • Error (0) No replication sessions started since server startup when writes are expected
  • Lag above your chosen threshold for consecutive checks
  • replication-lag-time reported as unavailable
  • Failed or incomplete initialization (Last Init Status not showing success)
  • Significant clock skew between hosts
  • A command execution failure or invalid JSON from a monitoring script

Do not alert solely on numeric code zero or non-zero; always interpret the complete status message. Avoid alerting solely because:

  • Update In Progress is temporarily TRUE
  • The last update timestamp is old in an idle directory
  • One transient busy response occurs in a multi-supplier topology

Set warning and critical thresholds from your own baseline. A directory with sub-second normal lag might alert at five seconds; a WAN-linked topology might tolerate minutes.


Triage abnormal monitoring results

Symptom First checks
Invalid credentials .dsrc, bind DN, password file path, file permissions
Replica unavailable DNS, port, TLS, dsctl instance status, firewall
Lag increasing Agreement errors, schedules, busy replicas, network delay
Out of synchronization Supplier and consumer CSNs in Replication Status, pending updates, errors log
Lag unavailable Remote status access, credentials, initialization state
Repeated replica busy Competing suppliers and session timing; review release timeout
Missing historical events Access-log retention, time range, log directory paths
Data still differs at zero lag Run ds-replcheck online between the two URLs

Use the symptom table to identify the affected layer, then continue with the relevant agreement, initialization, changelog, or topology-repair procedure.


References


Summary

Pick repl-agmt status for one agreement, replication status for all outgoing agreements on a host, and replication monitor for a topology-wide live report. Configure [repl-monitor-connections] and [repl-monitor-aliases] in ~/.dsrc so unattended runs do not prompt for passwords. Read Replication Status, lag, and CSN fields together — zero lag is not a substitute for ds-replcheck. Use replication lag-report on retained access logs for historical lag analysis, and wire dsctl healthcheck --check replication or JSON monitor output into your alerting pipeline.


Frequently Asked Questions

1. What is the difference between replication monitor and replication lag-report?

replication monitor walks the live topology and reports current agreement status and CSN-based lag from running replicas. replication lag-report analyzes historical access logs to calculate hop-by-hop and global lag for past changes. Use monitor for operational health checks; use lag-report for trend analysis and post-incident review.

2. Does zero replication lag prove the replicas contain identical data?

No. Zero lag means the supplier and consumer CSNs match for the monitored path, which indicates replication caught up on recent changes. It does not establish entry-level identity. Use ds-replcheck online for a deeper comparison, preferably during a quiet period, and ignore attributes intentionally excluded by fractional replication.

3. Why does repl-agmt status show Error (0) on success?

Error (0) means Directory Server recorded error code zero; Error is a legacy label. Read the complete message before deciding that the agreement is healthy. Incremental update succeeded confirms a successful session, while No replication sessions started since server startup does not prove that updates have flowed. Always check Replication Status and Replication Lag Time as well.

4. When is replication lag reported as unavailable?

Lag is unavailable when the monitoring tool cannot read consumer status — often because of network failure, invalid credentials, an uninitialized consumer, or a remote replica that is offline. Fix connectivity and credentials before you treat the agreement as healthy.

5. Is replication lag-report available in every 389 DS package?

No. The CLI lag analyzer was added in 389 Directory Server 3.1.3 and improved in 3.2.0. Run dsconf instance replication lag-report --help on your host to confirm the subcommand exists before you rely on it in automation.
Deepak Prasad

R&D Engineer

Founder of GoLinuxCloud with more than 15 years of expertise in Linux, Python, Go, Laravel, DevOps, Kubernetes, Git, Shell scripting, OpenShift, AWS, Networking, and Security. With extensive …