Projection catch-up and rebuilds

A projection can fall behind after a worker stops, SQL becomes unavailable, or a new view is introduced. Treat each case as explicit work with a source, destination, progress record, and failure policy. Restoring an actor's state does not repair all its read models automatically.

Distinguish the tasks

The same pure mapper can support more than one task, but a rebuild should not run business orchestration or external notification providers just because it reads historical events.

Bound one work unit

The example worker reads one page of verified rows, applies each idempotently, and advances its partition cursor. The application interfaces own verified source reads and durable compare-and-swap progress.

public static class ProjectionRepair
{
    public static async Task ProcessPageAsync(string partition, IRepairFeed feed,
        IRegistrationCardWriter writer, IRepairProgress progress, CancellationToken cancellationToken)
    {
        string? cursor = await progress.LoadAsync(partition, cancellationToken);
        RepairPage page = await feed.ReadAsync(partition, cursor, 100, cancellationToken);
        if (page.Items.Count > 100) throw new InvalidDataException("Repair page exceeded its bound.");
        foreach (RepairItem item in page.Items)
            await writer.ApplyAsync(item.Card, item.Source, cancellationToken);
        // A crash before this point repeats the page; the writer deduplicates source records.
        if (!await progress.AdvanceAsync(partition, cursor, page.NextCursor, cancellationToken))
            throw new InvalidOperationException("Another worker advanced this repair partition.");
    }
}

A crash before cursor advancement repeats the page. The projection writer's retained source identities make that repetition safe. The compare-and-swap failure tells the worker that another process advanced the partition; it cannot silently overwrite that progress.

This helper is one bounded work unit. A host scheduler supplies worker lifecycle, admission, leases where needed, time limits, backoff, and metrics. The interface implementations must check that the source and cursor belong to the expected partition and that a returned page cannot exceed its bound.

Keep serving and recovery separate

Avoid startup routines that scan every registration to reconstruct global read state. Their cost grows with application history, and one bad stream can delay global availability.

Startup can check required infrastructure and establish consumers. Repair workers can advance affected partitions in the background. The relevant endpoint should know whether its view is ready, catching up, unavailable, or quarantined. Do not represent missing projection data as a confidently empty catalogue.

Introduce a projection version

For a substantial change, create a new consumer identity and target schema, such as a second registration-card version. Populate it from verified source history, catch up with live traffic, compare the query behavior, then promote the readers. Keep a rollback or forward-fix plan based on compatible schemas and retained source data.

One useful cutover records a source boundary, backfills through that boundary, applies later live work, and checks the remaining lag before switching readers. The mechanism must fit your provider: a multi-stream application may need per-stream positions or committed operation groups rather than one global integer.

Do not clear an existing consumer's inbox and point the same identity at a new schema without deciding what delayed deliveries and old acknowledgements mean.

Invalid data and access changes

Transient connection failures can be retried. A source that cannot be decoded into the expected event schema needs an identifiable quarantine outcome and repair decision. Continue independent partitions where the provider permits it.

Current deletion and access fences also apply to rebuilt views. Historical ownership alone does not grant a restored view permission to expose deleted or inaccessible data. Coordinate projection repair with the application's retention and erasure policy.

Retain useful progress

Track the projection version, source partition, cursor, attempt count, last successful advance, and failure category. Keep private domain payloads out of routine progress logs. Monitor the oldest pending work as well as counts; a small permanently stuck partition can be more serious than a briefly large queue.

Continue with replay and live events, projection design, and observability.