Measure durations with the monotonic clock

Linux has several clocks, and mixing them up is a reliable way to ship a bug that only appears in production. CLOCK_REALTIME is the wall clock: NTP, settimeofday and leap seconds move it forwards or backwards. CLOCK_MONOTONIC counts from boot and is never stepped, though it does follow frequency adjustments and it stops during suspend — CLOCK_BOOTTIME includes the suspend time, and CLOCK_MONOTONIC_RAW skips adjustment entirely.

The rule that keeps this simple: intervals, timeouts, backoff and deadlines use CLOCK_MONOTONIC; anything a human or a peer will read — log timestamps, database rows, TLS validity, leases — uses CLOCK_REALTIME, and always through one helper rather than at each call site. Most languages already default to the right thing: time.monotonic() in Python, Instant in Rust, time.Since on a value from Go’s time.Now() are monotonic, while date(1), datetime.now() and SystemTime::now() are not.

The failure always has the same shape. A sync step rewinds the clock, code that stored a wall-clock timestamp subtracts it later, and the result is a negative interval: a retry loop that backs off further into the past, or an expiry that never fires. Journald keeps both, which is how you investigate it afterwards:

journalctl -u example.service -o short-monotonic --since "-30min"
journalctl -o json | jq -r '.__MONOTONIC_TIMESTAMP'

Steps and slews also have different operational consequences. systemd-timesyncd steps the clock by default; chrony and ntpd slew, and the kernel caps slewing at 500 ppm, so a clock a second out takes about half an hour to walk back. That is the right behaviour for a busy machine and the wrong one for a failover pair, because a step hits every node simultaneously and two peers can briefly both believe they are primary. And if you ever need the wall clock virtualised: time namespaces offset CLOCK_MONOTONIC and CLOCK_BOOTTIME only, so unshare --time will not fake a timezone and a container’s CLOCK_REALTIME is the host’s.

CLOCK_REALTIME_COARSE is the cheap one — wall-clock accurate to the timer tick, cached in a page the kernel keeps updated for you — and it is the right clock for stamping millions of events where microseconds are noise. Finally, timedatectl on a dual-boot box is worth reading once: another operating system will happily leave the RTC in local time for the Linux side to misread.