Summary#

Event sourcing systems fail less often because “events cannot be stored” and more often because the operational contract around the event log drifts: old snapshots no longer match current upcasters, projections lag behind writes, concurrent appends race on stale stream revisions, and replay jobs accidentally cross boundaries that should have been explicit. A robust design treats the event log as immutable but treats every derived artifact—snapshots, read models, subscriptions, projections, caches, indexes—as disposable, versioned, and observable.

The reusable rule is: only the event stream is authoritative; everything else needs a rebuild, versioning, idempotency, and boundary policy.

Key Points#

• Snapshot / upcasting drift
• Snapshots are derived state, not source of truth.
• If event schemas evolve but snapshots are not versioned or invalidated, a restored aggregate may represent a state that could no longer be produced by replaying the current upcasted event stream.
• Upcasters should be tested against old event fixtures, but snapshot compatibility must be tested separately.

• Safer policies:

  • include snapshot schema/version metadata;
  • invalidate snapshots when aggregate logic or upcasting rules change;
  • periodically verify “snapshot + tail events” equals “full replay from genesis”;
  • avoid keeping snapshots forever without a retirement plan.

• Event schema evolution and upcasting

• Upcasting is useful when old persisted events must be transformed into shapes expected by current code.
• Failure modes include:
  • non-deterministic upcasters;
  • upcasters that depend on external mutable data;
  • missing coverage for rare historical event variants;
  • changing event meaning rather than only event representation.

• Upcasting should preserve semantic intent. If a past event meant something different from today’s command model, a compensating event or explicit migration may be safer than pretending the old event was always the new concept.

• Projection lag

• In CQRS/event-sourced systems, read models are usually eventually consistent.
• Projection lag becomes a failure when callers assume read-your-writes consistency without an explicit mechanism.
• Common symptoms:
  • user writes succeed but UI immediately reads stale state;
  • downstream workflows trigger from old projections;
  • rebuilds temporarily expose partial or mixed-version read models.

• Safer policies:

  • expose projection position/checkpoint where useful;
  • design UX and APIs around eventual consistency;
  • use idempotent projection handlers;
  • rebuild into a shadow table/index, then atomically switch readers;
  • monitor subscription/checkpoint lag as an SLO, not just as a background metric.

• Optimistic concurrency

• Event streams commonly rely on expected stream revision/version checks.
• A command handler reads an aggregate at revision N, decides, then appends only if the stream is still at N.
• Failure modes:
  • ignoring expected revision and accidentally allowing lost updates;
  • retrying commands blindly and producing duplicate or semantically invalid events;
  • using a single hot stream that causes constant concurrency conflicts;
  • treating concurrency exceptions as infrastructure failures rather than domain conflicts.

• Safer policies:

  • append with expected revision;
  • make commands idempotent where possible;
  • distinguish retryable technical errors from real domain conflicts;
  • consider stream partitioning for hot aggregates;
  • include command/request IDs when duplicate command submission is possible.

• Replay boundary design

• Replays are not just “run all handlers again.”
• Projection replay can accidentally resend emails, charge payments, republish integration messages, or mutate external systems if handlers mix pure projection logic with side effects.
• Explicit replay boundaries should define:
  • which handlers are replay-safe;
  • which projections can be rebuilt from which event ranges;
  • whether replay starts from stream beginning, a checkpoint, a snapshot, or a migration boundary;
  • how duplicate event handling is detected;
  • whether external side effects are disabled, mocked, or routed to a separate channel during replay.

• A useful pattern is to separate:

  • decision replay: reconstruct aggregate state;
  • projection replay: rebuild read models;
  • integration replay: generally avoided or tightly controlled because external side effects may not be reversible.

• Duplicate handling

• At-least-once delivery is common in subscription/projection mechanisms.
• Projection handlers should assume duplicates and out-of-order operational effects may occur unless the platform gives stronger guarantees.

• Use event IDs, stream positions, global positions, or per-projection checkpoints to deduplicate.

• Operational checklist

• Keep old event fixtures in tests.
• Test full replay regularly.
• Test snapshot restore plus replay of subsequent events.
• Version snapshots separately from events.
• Track projection lag and replay progress.
• Make projection handlers idempotent.
• Use expected stream revision on append.
• Define replay-safe and replay-unsafe handlers.
• Treat read models as rebuildable artifacts.
• Document consistency guarantees per API endpoint or workflow.

Cautions#

• “Snapshot drift” is a practical failure mode, but terminology varies by source. Some documentation discusses snapshotting, event versioning, and replay separately rather than using the exact phrase “snapshot drift.”
• EventStoreDB, Axon Framework, and Microsoft architecture guidance use different vocabulary for streams, revisions, subscriptions, processors, projections, and checkpoints. The concepts are portable, but implementation details are framework-specific.
• Projection lag is not always a bug; it is often an accepted consequence of eventual consistency. It becomes a defect when the system or users require stronger consistency than the design provides.
• Optimistic concurrency prevents some lost updates, but it does not automatically solve duplicate command submission, non-idempotent side effects, or poorly chosen aggregate boundaries.
• Replaying events through handlers that perform external side effects is dangerous unless those handlers are explicitly replay-aware or disabled during replay.

Sources#

• https://learn.microsoft.com/en-us/azure/architecture/patterns/event-sourcing
• https://developers.eventstore.com/server/v24.10/event-sourcing.html
• https://developers.eventstore.com/clients/grpc/appending-events.html
• https://developers.eventstore.com/clients/grpc/subscriptions.html
• https://docs.axoniq.io/axon-framework-reference/4.10/events/event-versioning/
• https://docs.axoniq.io/axon-framework-reference/4.10/events/event-processors/
• https://docs.axoniq.io/axon-framework-reference/4.10/tuning/event-snapshots/

Related#

• Event Sourcing Snapshot and Schema-Evolution Failure Modes: Upcasters, Replay Determinism, Snapshot Invalidation, and Projection Rebuild Cutover
• Write Boundary Leakage, Projection Lag UX, Command Idempotency, and Authorization Drift
• CQRS Read-Model Projection Failure Modes: Ordering, Idempotent Replay, Poison Events, and Rebuild Cutover

Sagwan Revalidation 2026-07-07T21:35:56Z#

• verdict: ok
• note: 수치·링크 의존 없이 현재 이벤트 소싱 운영 원칙과 부합함

Sagwan Revalidation 2026-07-09T19:24:14Z#

• verdict: ok
• note: 이벤트소싱 운영 원칙과 실패 모드는 현재 practice와도 대체로 일치함

Sagwan Revalidation 2026-07-11T11:42:55Z#

• verdict: ok
• note: 이벤트소싱 운영 원칙으로 현재도 유효하며 갱신할 수치·링크 없음.

Sagwan Revalidation 2026-07-13T06:32:12Z#

• verdict: ok
• note: 일반 원칙 중심이라 최신 관행과 충돌하거나 갱신할 수치·링크가 없음

Sagwan Revalidation 2026-07-15T04:55:39Z#

• verdict: ok
• note: 일반적 운영 원칙 위주라 최근 관행과 충돌 없이 재사용 가능함

Sagwan Revalidation 2026-07-17T06:11:48Z#

• verdict: ok
• note: 개념·권장안 모두 현재 이벤트소싱 실무와 충돌 없이 유효함

Sagwan Revalidation 2026-07-19T07:20:06Z#

• verdict: ok
• note: 이벤트소싱 운영 원칙과 권장안이 현재 practice와 충돌하지 않음.

Sagwan Revalidation 2026-07-21T09:05:59Z#

• verdict: ok
• note: 수치·링크 없는 일반 원칙으로 현재 이벤트 소싱 practice와도 부합함