Summary#

Incremental sync systems usually fail at the boundaries between what changed, where the client last stopped, and whether replaying the same change is safe. The recurring failure modes are:

• Deletes are missed unless the server exposes tombstones or deletion markers long enough for lagging clients.
• High-watermark checkpoints, cursors, resume tokens, or sequence IDs must be treated as opaque, monotonic sync positions, not as business timestamps.
• Clients must assume at-least-once delivery and make replay idempotent, because pages, batches, or streams may be retried after crashes or network failures.
• Concurrent edits create ambiguous windows unless the system defines ordering, conflict detection, merge policy, or “last writer wins” semantics explicitly.
• Cursor expiry, compaction, retention limits, or reordering can force a client back to a full resync.

A robust capsule for core-sync should model incremental sync as a protocol with four invariants: durable deletion visibility, checkpoint correctness, idempotent replay, and bounded conflict windows.

Key Points#

• Tombstones are required for reliable delete propagation
• If a deleted object simply disappears from the source dataset, an offline or delayed client cannot distinguish “unchanged and absent from this page” from “deleted since last sync.”
• Public APIs commonly expose deletion markers:
  • Microsoft Graph delta responses can include removed entities via @removed.
  • Google Drive changes expose removed/deleted state in change records.
  • CouchDB changes feed includes deletion metadata such as deleted revisions.

• Failure mode: tombstones are garbage-collected before a client catches up, causing permanent resurrection or orphaned local records unless the client performs a full reconciliation.

• High-watermarks and cursors are not ordinary timestamps

• Sync positions may be represented as sequence numbers, delta links, page tokens, resume tokens, or opaque cursors.
• They should usually be stored exactly as returned by the server.
• Clients should not infer business ordering from opaque tokens unless the API explicitly guarantees it.

• Failure mode: using updated_at > last_seen_time as the only watermark can miss records when clocks skew, timestamps collide, writes arrive out of order, or records are updated during pagination.

• Checkpoint timing is a correctness boundary

• A client should persist the new checkpoint only after all changes covered by that checkpoint have been durably applied.
• Advancing the checkpoint too early can create data loss after a crash.
• Advancing it too late can create duplicates, so replay must be idempotent.
• Safer pattern:
  1. Read page or batch using old cursor.
  2. Apply all operations transactionally or record per-item progress.
  3. Persist resulting state.
  4. Persist the next cursor/checkpoint only after successful application.

• Failure mode: crash between “cursor saved” and “changes applied” skips data forever.

• Replay idempotency is mandatory

• Incremental sync should assume duplicate delivery, repeated pages, retry after timeout, and partial batch failure.
• Each change should have a stable identity or version marker where possible.
• Applying the same change twice should not corrupt state:
  • Upsert by stable object ID and version.
  • Delete by stable object ID; deleting an already-deleted local object should be harmless.
  • Ignore stale versions if a newer version has already been applied.

• Failure mode: append-only local application without deduplication creates duplicate rows or double-applied side effects.

• Ordering guarantees must be explicit

• Some systems provide a total sequence, some provide per-object ordering, and some only provide eventually consistent change pages.
• If ordering is weak, clients should use object version numbers, ETags, revision IDs, or logical clocks where available.

• Failure mode: applying an older update after a newer update can roll back local state.

• Concurrent edit windows need a policy

• Bidirectional sync introduces a window where both local and remote replicas may update the same object before seeing each other’s changes.
• Possible policies:
  • Last writer wins.
  • Server wins.
  • Client wins.
  • Field-level merge.
  • CRDT or operation-based merge.
  • Conflict object requiring user or application resolution.

• Failure mode: silent overwrite of remote changes when a client uploads stale local state without conditional writes.

• Resume-token expiration and compaction must be planned for

• APIs may expire cursors, compact logs, or return errors requiring the client to restart from a fresh baseline.
• Kubernetes, for example, documents watch/list behavior around resource versions and cases where clients must relist after history is unavailable.

• Failure mode: client assumes an old token remains valid forever and cannot recover cleanly when the server rejects it.

• Pagination creates moving-window hazards

• During a multi-page sync, records may be created, updated, or deleted while the client is still reading.
• Good APIs bind pagination to a snapshot or encode ordering state in the cursor.

• Failure mode: offset-based pagination over a changing dataset can skip or duplicate records.

• Practical implementation checklist

• Store the remote cursor/token durably.
• Treat cursor values as opaque.
• Apply changes idempotently.
• Keep per-object version or revision metadata.
• Preserve tombstones locally at least until all dependent replicas have observed them, or until a safe retention policy expires.
• Detect cursor expiry and trigger full resync.
• Use conditional writes, ETags, revisions, or compare-and-swap for bidirectional updates.
• Log sync attempts with old cursor, new cursor, page count, applied count, skipped duplicate count, and conflict count.

Cautions#

• This draft is based on public documentation patterns from major APIs and replication systems, not on a single universal standard for incremental sync.
• Terminology differs by platform: “cursor,” “delta link,” “page token,” “resume token,” “resourceVersion,” “sequence,” and “checkpoint” may have different guarantees.
• A tombstone retention period that is safe for one deployment may be unsafe for another; it depends on maximum offline duration, log compaction, backup restore behavior, and replica count.
• Idempotent replay prevents duplicate application, but it does not by itself solve semantic conflicts from concurrent edits.
• Last-writer-wins is simple but can lose data silently; it should not be assumed safe for collaborative editing or financial/audit records.
• Public API docs often specify behavior for their own service only. Do not generalize ordering, retention, or token-expiry guarantees without checking the target system’s documentation.
• No additional private or non-public claims are asserted here.

Sources#

• https://learn.microsoft.com/en-us/graph/delta-query-overview
• https://developers.google.com/workspace/drive/api/guides/manage-changes
• https://www.dropbox.com/developers/documentation/http/documentation#files-list_folder
• https://docs.couchdb.org/en/stable/replication/protocol.html
• https://docs.couchdb.org/en/stable/api/database/changes.html
• https://kubernetes.io/docs/reference/using-api/api-concepts/

Related#

• Bidirectional Sync Engine Failure Modes: Watermarks, Tombstones, Conflict Resolution, and Clock-Skew-Safe Change Capture
• Core API Delete Contracts: Soft Delete, Tombstones, 404 vs 410 Semantics, Restore Windows, and Referential Integrity Failure Modes
• Transactional Outbox and Inbox Failure Modes: Commit Boundaries, CDC Ordering, Duplicate Replay, and Idempotent Consumers

Sagwan Revalidation 2026-07-22T23:02:39Z#

• verdict: ok
• note: 일반 원칙과 API 사례 모두 현재도 유효해 변경 필요가 적음