Skip to content

Migrating Between EventSourcingDB Instances

This guide explains how to migrate data from one EventSourcingDB instance to another. It covers typical scenarios such as infrastructure transitions, GDPR compliance, and selective data retention. You'll learn how migrations work, what data is included in a backup, and what design constraints apply when restoring into a new instance.

This is not only a technical process, but also a conceptual one: migrating event streams means reestablishing the system's truth in a new place. It requires careful preparation, awareness of the system's invariants, and a clear understanding of what is – and isn't – allowed.

Migration Workflow

EventSourcingDB supports migrations via its built-in backup and restore functionality. A backup captures all relevant data from an instance and stores it in newline-delimited JSON format. This includes:

  • All events, in the order they were written
  • All registered event schemas

Snapshots are implicitly included because they are modeled as regular events. There is no separate format or mechanism for exporting them. If your system uses snapshots, they will be backed up and restored automatically.

Licenses are not included in the backup. These must be configured separately on the target instance, either via a license file or inline string. The same applies to runtime configuration such as API tokens, TLS certificates, and port settings – these are not part of the database itself.

A typical migration consists of the following steps:

  1. Put the source instance into read-only mode to prevent new writes during backup.
  2. Create a backup using the HTTP API.
  3. Optionally process the backup file to filter or transform events.
  4. Start a new, empty instance in lockdown mode.
  5. Restore the filtered backup into the new instance.
  6. Resume normal operation on the target system.

This approach ensures a clean and deterministic transfer of data between instances, with no race conditions or data loss.

Design Constraints and Limitations

There are several important limitations to be aware of:

  • The target instance must be completely empty. It is not possible to restore into an instance that already contains data.
  • Events are immutable and strictly ordered. Individual events cannot be removed, rewritten, or selectively restored. You must restore a consistent sequence that starts from ID zero.
  • Backups and restores operate at the database level, not per subject or stream. You cannot extract a single stream or restore only part of a hierarchy.
  • There is no mechanism for merging data from multiple backups or for appending new events to an existing instance via restore. The restore process assumes full control over the target system.

Because of these constraints, migration is most appropriate for clearly scoped, well-prepared scenarios — such as moving between environments, replacing infrastructure, or complying with data retention requirements.

GDPR and Data Minimization

One important use case for migration is compliance with regulations such as the General Data Protection Regulation (GDPR). Since EventSourcingDB does not support deletion of individual events, the only way to permanently remove or redact data is to export a backup, filter or anonymize it externally, and then restore it into a new instance.

This requires careful analysis of the events in question. You need to understand which fields contain personal data, and how to modify or remove them without breaking structural integrity or downstream projections. In some cases, replacing sensitive data with placeholders or hashes is sufficient. In other cases, the events may need to be excluded entirely – which affects ordering, projections, and any dependent logic.

EventSourcingDB does not provide built-in tooling for filtering backups. It is your responsibility to process the exported JSON file as needed. Because backups are stored in a simple, line-based format, this can typically be done using standard tools or scripts in your language of choice.

This strategy is sometimes referred to as crypto shredding or data thrashing — not because the database deletes the data itself, but because the restored version no longer contains the original secrets. Another option is to store only opaque identifiers in events and keep sensitive data in an external system. This makes it easier to comply with deletion requests by removing the external records alone.

Whatever approach you choose, the key point is that compliance must be designed into your system. Event-sourced architectures are not inherently incompatible with GDPR, but they require proactive modeling and clear separation of concerns to support compliance safely.

Operational Considerations

Before starting a migration, always ensure that the source system is in a stable state. Writing must be disabled, either via the read-only mode or by stopping the application that produces events. This guarantees that the backup reflects a consistent and complete snapshot of the system at a specific point in time.

The backup file can be large — especially in systems with millions or billions of events — but it is written in a streaming-friendly format that can be processed incrementally. You do not need to load the entire file into memory, and most filtering or transformation tasks can be done line by line.

Restoring into the target instance requires that it be in lockdown mode. This is a special startup mode in which no external writes or reads are allowed, and only the restore endpoint is active. Once the restore completes, the instance can be restarted in normal mode and begins serving requests as usual.

This design ensures that the restored state is trustworthy, deterministic, and uncorrupted by concurrent activity.

Planning for the Future

Even if you do not currently plan to migrate your instance, it is a good idea to design your events and domain model with migration in mind. Avoid storing large amounts of sensitive data directly in events unless necessary. Use versioned event types, well-defined schemas, and clear subject hierarchies to make it easier to analyze, transform, and filter your data later.

EventSourcingDB provides a solid and transparent foundation for durable event storage. But it is your responsibility to define what should be stored, how long it should be retained, and how it should be interpreted. Migration is not just a technical tool — it is a strategic option that supports maintainability, compliance, and long-term evolution.