Your Progress Bar Doesn't Belong in the Event Store¶

The progress bar works. It fills up, users can see how far along their contract review is, and nobody has complained. Then you open the event store and find 312 events for a single contract, 310 of which nobody would have written if that bar did not have to move.

Nothing is broken, and that is exactly what makes this hard to argue about. The system does what it should, the event names speak the language of the domain, and every one of those events was written on purpose. The unease is real anyway, and if you can't say where it comes from, the discussion ends before it starts. So: most of those events have a shelf life, and the event store is the one place in your system that doesn't.

The Chain That Leads There¶

Nobody decides one morning to fill the store with noise. The reasoning that gets you there is short, and every step in it sound.

A long-running process should show how far it has come. That progress is read from a projection, because in an event-sourced system a projection is how you read anything. Projections are updated by events. Therefore you need events – one when a clause check begins, one when it ends – and the bar has what it needs.

Four steps, no error in any of them. And once those events exist, they defend themselves rather convincingly: they have business-sounding names, and something visible in the product depends on them. Try removing ClauseCheckStarted and you will be told, correctly, that a feature people asked for would stop working.

This is why the pattern survives review: it doesn't look like a shortcut, it looks like Event Sourcing done properly.

A while ago, in Event Sourcing is Not For Everyone, we offered a test for telling domain events from raw data: Would anyone want to be notified about this specific occurrence? If yes, you're probably looking at an event. If no, it's probably just data. That test does good work. It sorts GPS coordinates and temperature readings out of the store, and it does it in one question.

Run the clause check through it. Would anyone want to be notified that clause 47 of 155 has just been examined? Yes. Someone is sitting in front of a screen waiting for exactly that.

On the face of it there is a consequence: a projection updates, a bar moves. This is no sensor reading without meaning of its own; it's a deliberate step in a deliberate process.

The test waves it through, and the test is not wrong. It is incomplete, because it asks whether anyone cares and asks it in the present tense. Progress is the clearest kind of fact that answers yes to that question and still doesn't belong in the store, because the interest it commands expires the moment the process ends.

You could push back here, because the original example justified its yes with "Multiple parts of the system care about this occurrence", and only the progress view cares about a clause check. But what gets quoted in design discussions is the question, not the sentence that followed it in one example. And the question is passed the moment the projection exists, because from then on something in the system genuinely does care – you built it to.

Three Shelf Lives¶

None of this is about disk space. We ran that math in 18 Months of Events Fit on Four Floppy Disks, and the conclusion stands: events are small, stores handle millions of them, and storage should almost never enter an architectural decision. The cost here isn't bytes, it's noise.

So the useful question isn't whether a fact matters. It's how long it matters, and there are three answers.

Some facts matter right now and not a second longer. The message that moves the bar one step is the clearest example: it is interesting on its way to the screen, and by the time it arrives the next one is already due. Nothing needs to store it at all.

Some facts matter for a while. They have to survive a crash so the run can pick up where it left off instead of starting over, and they have to survive a page reload so the bar doesn't reset to zero. But once the run is finished, their job is finished too.

And some facts matter forever. That a contract was reviewed on a particular day, against a particular rule, with a particular outcome, will not stop being true or useful. That is what the event store is for, and why EventSourcingDB has no expiry dates and no API that removes an event. The manifesto in Thinking in Events says as much: we do not erase the past, we add to it.

Now look at how most systems are laid out. There's memory, which dies with the process, and the event store, which keeps things forever. There's usually a read-model database too, but everything in it is derived and rebuildable from the events at any time – and how far a run has come is precisely what isn't.

So the middle horizon ends up with no designated home, and when something has to survive a crash, the store is the first thing that comes to mind. That isn't carelessness: where durability means the store, it's the obvious move.

State, Not History¶

What the progress view needs is not a stream of things that happened, but where the run stands – and that difference is the whole point.

The distinction isn't large versus small. If you want to show which clauses are done and which one is being worked on right now, the state is a list, and for a long contract a long one. That list is still state.

It has one current version, gets written over as the run proceeds, and only the latest is ever read. A history keeps every version that ever existed, in order, timestamped, and never lets go.

You can derive the state from the history – that's what a projection does, and usually it's a good deal. Here it isn't, because you pay for the derivation with permanence: to know where the run stands, you commit to remembering every step it took, forever.

So keep the state as state. There are three ordinary places to put it:

Process state you overwrite. A row per run, updated as it goes: which clauses are done, which one is in progress, how many are left. Survives a crash, supports resuming, and is deleted when the run ends.
A store that forgets on its own. A cache or key-value store with a time to live, where expiry is built in and nobody has to remember to clean up – exactly what the event store deliberately lacks.
Whatever your infrastructure already tracks. If the run goes through a queue, a scheduler, or a workflow engine, the progress and the resume point are already there. Then the fix isn't a new component, it's not writing the same information twice.

None of these is exotic, and that's rather the point. The middle horizon doesn't need clever technology. It needs to be noticed.

The first of them invites an objection, and a team doing Event Sourcing will raise it at once: a row per run is a second place that writes durably, and getting away from that is half the reason anyone adopts Event Sourcing. But the process row is not a second source of truth, because nothing about the review is decided from it. If it is lost or wrong, the run redoes some work and the bar starts over – the only durable claim about the review is the closing event.

What the Closing Event Carries¶

Skipping the per-clause events doesn't mean losing what the review found. The findings exist either way; the only question is where they're written down – and the closing event has room.

ContractReviewCompleted can carry which clauses were examined, which were flagged, and what the finding was for each. That's not a compromise but a better record than 310 separate events, because it can be read in one go by anyone asking what people actually ask later: what came out of the review of contract 4711? It makes for a wide event, which is fine – Don't Put That PDF in Your Event is about attachments, not structured domain information.

What genuinely goes away is the order in which the clauses were examined, and how long each took. Be deliberate about that, because it is the real loss – then notice how unlikely anyone is to ask. Nobody opens an event store to find out that clause 12 was checked before clause 47.

There is a second reason the per-clause events are redundant. "The contract was reviewed" already says that its clauses were examined; that's what reviewing a contract means. Where the upper level takes in the lower one by definition, the lower one is contained in its result – which holds here, and does not hold everywhere, so it is worth checking rather than assuming. Once the closing event is written, every step anyone will ask about later has been said twice, and only one of the two versions will ever be read.

Private Journal, Public Announcement¶

There's a second way to arrive at the same line, and it has nothing to do with time but with direction.

In One Database to Rule Them All we put it like this: your domain events are your private journal, your integration events are your public announcements, and the two should be kept apart. Domain events can be fine-grained, awkwardly specific, and shaped entirely around what your write model needs, precisely because nobody outside has to understand them. That's the freedom of keeping them inside.

That post drew the line between services. The same line runs between your service and its own front end. The browser is outside too.

Progress information sits firmly on the outer side: it exists to be shown to someone, and no domain decision rests on it. That explains the pattern differently than the chain at the top of this post, and it reaches further.

Observing events is the obvious way to keep something in sync in an event-sourced system. So when the browser needs to stay in sync, the same reflex fires – and to observe something, something has to be written. They passed for facts, and nothing in the test stopped them. But what actually put them there is that the store was the transport lying around.

The consequence is worth spelling out: your internal model is now the wire format of your user interface. Rename an event, split it, change its payload, and something on a screen breaks. That is a coupling nobody chose and nobody wrote down.

The alternative is to send the two kinds of thing differently. Progress is pushed to the client with its content, over the ordinary server-push channel you would use anyway and fed straight from the process state, so nothing has to be written for it to travel. A change to the domain is announced without its content – a signal that something is new, after which the client asks for what it needs through the read model built for it. For the domain: push to pull.

Observing keeps its place; it is still how your own components learn that something happened. What changes is what leaves the building: the observer inside turns a domain event into a signal, and the signal is what the browser gets to see. The journal stays inside, and the outside is told only that there's something new to look at.

But What If I Need It Later?¶

The strongest objection to all of this is a good one, and it deserves better than reassurance. In Naming Events Beyond CRUD we called it a one-way door: you can always combine fine-grained events into a coarser view, but you can never split a coarse event into finer ones after the fact. Write too little and the information is gone for good.

That's true. What's less often said is that there is a second one-way door, facing the other way: everything you write is permanent as well.

Every event type has to be versioned as the domain moves under it – see Versioning Events Without Breaking Everything. Every event is replayed each time a projection is rebuilt, and every event is something a person debugging at two in the morning has to read past. And getting personal data back out of a store designed never to forget means migrating the whole thing into a fresh instance, which is why our GDPR Compliance guide opens by telling you to think before you write.

So both directions cost something: there's no safe default and no reflex that gets you out of deciding. The answer isn't fewer events. It's the events someone will ask about.

If the sequence genuinely matters in your domain – because an auditor will want the order, or because the duration per clause is the product – then model it, deliberately. If what matters is the outcome, model the outcome. Fine-grained always meant saying precisely what happened. It never meant saying it more often.

Two Events, and the Bar Still Fills¶

Back to the 312. After the change there are two: one that opens the review and names its scope, one that closes it with the findings. Everything in between lives in the process state, is pushed straight to the browser while it's relevant, and is thrown away when the run ends. That it came out at two is the result, not the aim – the aim was to write down only what someone will ask about later.

And the progress bar still fills, exactly as before. Nothing about the user-facing feature was the problem – on a long review it's the right thing to build, and showing which clause is being examined is a courtesy to someone who would otherwise stare at a spinner. Only its address was ever in question.

The question worth carrying away is short enough for a design discussion. Not "Would anyone want to be notified about this specific occurrence?" – everything someone is waiting for passes that one – but "will anyone still want to know about this a year from now?" Facts that survive it belong in the store. The rest needs a home only for as long as it matters, for a while or not at all, and giving it that place keeps the store worth reading.

If you look at your own events and can't say which of them anyone will still ask about a year from now, we'd be glad to go through them with you – write to us at hello@thenativeweb.io.