In the first article, we removed the polling tax.

Instead of repeatedly asking the database:

“Did anything change?”

we captured relevant mutations atomically using a Transactional Outbox.

In the second article, we solved the next problem:

“How do we move those events out of MySQL without polling again?”

We followed the MySQL Binary Log, used GTID-based positioning, and built a lightweight Go CDC streamer.

Then, in the third article, we introduced another architectural boundary:

“How do we turn low-level database changes into stable events and distribute them without tightly coupling producers and consumers?”

We transformed database changes into CloudEvents v1.0 and introduced a durable messaging layer such as RabbitMQ or NATS JetStream.

At this point, the architecture looked mature.

But distributed systems have a habit of asking one final question:

“What happens when the same event arrives twice?”

That question changes everything.

Because At-Least-Once delivery means duplicates are not an exceptional condition. They are part of the design.

This article completes the architecture by introducing two final tactical patterns:

Idempotent Consumers

and

Anti-Corruption Layers (ACLs)

We will also take the last step: moving read-heavy workloads, including AI agents and public APIs, away from the legacy transactional ERP and onto optimized read models.

The result is a system where:

  • the ERP remains the source of transactional truth,

  • downstream consumers become resilient to duplicate delivery,

  • legacy terminology is isolated behind explicit boundaries,

  • and AI agents can query optimized projections without touching the production ERP.

The Friday 6 PM Incident#

It happened on a Friday at 6:00 PM.

A customer was charged twice.

At the same time, stock for one of the best-selling products went negative.

The investigation eventually pointed to a classic distributed-systems failure.

The consumer had successfully processed the event.

Then the process crashed.

It died before acknowledging the message.

When the consumer came back online, the broker redelivered the message.

That behavior was not a broker bug.

It was the expected consequence of At-Least-Once delivery.

RabbitMQ explicitly documents that messages can be redelivered after failures and recommends that consumers be designed to handle messages they have already seen. Publisher retransmissions can also create duplicates when a confirmation is lost during a network failure.

The actual sequence was:

The failure was not that the broker delivered the message twice.

The failure was that the consumer assumed:

“I will only ever see this event once.”

That assumption does not survive distributed systems.

At-Least-Once Changes the Design#

At-Least-Once delivery gives us a very specific contract:

The message should not be silently lost,
but it may be delivered more than once.

That means the consumer has to be designed around the possibility of:

The goal is therefore not to make the transport magically exactly-once.

The goal is to make the business effect happen once.

That is a very different problem.

Conceptually:

The Wrong Way to Deduplicate#

The first implementation many teams write looks like this:

SELECT EXISTS(
    SELECT 1
    FROM processed_events
    WHERE event_id = ?
);

If the event does not exist:

process event
insert event_id

It looks reasonable.

But there is a race.

Two consumer instances can receive the same event at nearly the same time:

The problem is the gap between:

check

and:

write

That gap is enough for two workers to make the same decision.

Idempotency Must Be Atomic#

The better pattern is:

Make deduplication part of the same transaction that applies the business effect.

The database should decide whether this event is new.

If the event is new:

  1. register the event,

  2. apply the business mutation,

  3. commit both together.

If the event already exists:

  1. do nothing,

  2. acknowledge the message.

The desired flow becomes:

The key is the unique constraint.

For example:

CREATE TABLE processed_events (
    event_id VARCHAR(255) PRIMARY KEY,
    processed_at TIMESTAMP(6) NOT NULL
);

The primary key itself becomes the concurrency control mechanism.

Why the CloudEvent ID Matters#

In the previous article, we introduced CloudEvents.

One field now becomes especially important:

{
  "specversion": "1.0",
  "type": "com.example.erp.order.created",
  "source": "/erp/orders",
  "id": "6a9d0e8d-9f1a-4b0d-b4e1-df75d0c8f7bd",
  "data": {
    "order_id": "ORD-92831"
  }
}

CloudEvents requires producers to ensure that the combination of source and id is unique for each distinct event. Consumers may use identical source and id values to identify duplicates.

That gives the consumer a stable identity to work with.

The event can be retried.

Redelivered.

Replayed.

Moved through another transport.

And the consumer can still answer:

“Have I already processed this event?”

The Correct Go Consumer#

The consumer should not perform:

as three independent operations.

Instead, we want one transaction.

A simplified version:

package consumer

import (
	"context"
	"database/sql"
	"encoding/json"
	"errors"
	"fmt"
)

type OrderConsumer struct {
	db            *sql.DB
	domainService OrderDomainService
}

type CloudEvent struct {
	SpecVersion string          `json:"specversion"`
	ID          string          `json:"id"`
	Source      string          `json:"source"`
	Type        string          `json:"type"`
	Data        json.RawMessage `json:"data"`
}

func (c *OrderConsumer) ProcessEvent(
	ctx context.Context,
	eventBytes []byte,
) error {

	var event CloudEvent

	if err := json.Unmarshal(eventBytes, &event); err != nil {
		return fmt.Errorf("decode CloudEvent: %w", err)
	}

	if event.ID == "" || event.Source == "" {
		return errors.New("CloudEvent source and id are required")
	}

	tx, err := c.db.BeginTx(ctx, nil)
	if err != nil {
		return err
	}

	defer tx.Rollback()

	// Register this event atomically.
	// A UNIQUE constraint on (source, event_id)
	// turns duplicate delivery into a safe no-op.
	result, err := tx.ExecContext(ctx, `
		INSERT INTO processed_events (
			source,
			event_id,
			processed_at
		)
		VALUES (?, ?, CURRENT_TIMESTAMP(6))
		ON DUPLICATE KEY UPDATE event_id = event_id
	`,
		event.Source,
		event.ID,
	)

	if err != nil {
		return fmt.Errorf("register event: %w", err)
	}

	rows, err := result.RowsAffected()
	if err != nil {
		return err
	}

	// The event already existed.
	if rows == 0 {
		return nil
	}

	// Anti-Corruption Layer:
	// translate the external event into the
	// consumer's internal domain model.
	order, err := c.translateToDomain(event.Data)
	if err != nil {
		return fmt.Errorf("ACL translation failed: %w", err)
	}

	// IMPORTANT:
	// the domain operation must participate in the same
	// database transaction for the atomicity guarantee.
	if err := c.domainService.ApplyTx(ctx, tx, order); err != nil {
		return fmt.Errorf("apply domain event: %w", err)
	}

	if err := tx.Commit(); err != nil {
		return fmt.Errorf("commit transaction: %w", err)
	}

	return nil
}

The important detail is not the exact SQL syntax.

The important property is:

If the business operation fails, the processed-event record must not remain committed.

Otherwise a retry would be incorrectly discarded.

The Anti-Corruption Layer#

Idempotency solves one problem.

But there is another architectural problem.

Legacy systems contain legacy language.

Imagine the ERP calls an order:

fktscxx001

and uses fields such as:

cod_art
alm_id
sit_tran
fec_ped
imp_tot

Those names may make perfect sense to the original team.

They should not leak into the new domain.

This is where the Anti-Corruption Layer (ACL) becomes useful.

The ACL translates between two models:

The new service should speak its own language.

For example:

The legacy model stays outside the domain boundary.

Why an ACL Is More Than a Mapper#

It is tempting to think:

“The ACL is just a DTO converter.”

It can be much more important than that.

The ACL can isolate:

  • legacy field names,

  • legacy status codes,

  • inconsistent identifiers,

  • undocumented business rules,

  • unit conversions,

  • null semantics,

  • historical quirks,

  • and incompatible assumptions.

For example:

type LegacyOrder struct {
	CodArt   string `json:"cod_art"`
	AlmID    string `json:"alm_id"`
	SitTran  string `json:"sit_tran"`
	ImpTotal int64  `json:"imp_tot"`
}

The domain should never have to know about SitTran.

Instead:

type Order struct {
	ID             string
	ProductID      string
	WarehouseID    string
	Status         OrderStatus
	TotalCents     int64
}

And the ACL performs the translation:

func (c *OrderConsumer) translateToDomain(
	data []byte,
) (Order, error) {

	var legacy LegacyOrder

	if err := json.Unmarshal(data, &legacy); err != nil {
		return Order{}, err
	}

	status, err := mapLegacyStatus(legacy.SitTran)
	if err != nil {
		return Order{}, err
	}

	return Order{
		ID:         legacy.CodArt,
		ProductID:  legacy.CodArt,
		WarehouseID: legacy.AlmID,
		Status:     status,
		TotalCents: legacy.ImpTotal,
	}, nil
}

The point is not the example mapping.

The point is that legacy corruption stops at the boundary.

The Full Consumer Boundary#

The complete consumer now looks like:

The consumer has become an anti-corruption boundary.

It receives external events.

It translates them.

It protects itself against duplicate delivery.

It applies domain logic.

And then it maintains its own local read model.

CQRS Enters the Picture#

Now we reach the final architectural step.

The AI agent does not need to execute:

SELECT *
FROM fktscxx001
WHERE art_id = ?

against the ERP.

The agent usually needs a very different shape of data.

For example:

{
  "product_id": "12345",
  "warehouse": "MAD-01",
  "available_stock": 42,
  "last_updated": "2026-09-10T15:45:00Z"
}

The application can maintain this projection continuously from events.

Conceptually:

This is where CQRS becomes useful.

The write model can remain optimized for transactional correctness.

The read model can be optimized for:

  • search,

  • APIs,

  • dashboards,

  • analytics,

  • AI retrieval,

  • reporting,

  • and high-volume queries.

The two no longer need to have the same shape.

AI Agents Should Not Query the ERP#

This becomes especially important with AI.

An AI agent can generate unpredictable query patterns.

It may ask:

Show me all products with low inventory
in warehouses near Madrid
that had unusually high sales
during the last seven days.

You do not want that query hitting the same transactional database that processes checkout.

Instead:

The AI workload is now separated from transactional traffic.

The ERP remains responsible for truth.

The read model is responsible for serving queries.

From Synchronous Reads to Replicated State#

The key transformation is:

versus:

Or visually:

Now the AI agent is no longer a dependency of the transactional system.

It is a consumer of replicated state.

That is a much healthier architecture.

Latency Becomes a Different Problem#

Once the read model exists locally, the application no longer needs to wait for the ERP network round trip for every query.

For the workloads measured in this implementation, the result was:

Legacy path P50:        1,247 ms
Read-model P50:            32 ms

That represents a measured 97.4% reduction in P50 latency for the specific workload and deployment described here.

The point is not that every CQRS architecture will achieve these exact numbers.

The point is that removing the remote transactional database from the read path can radically change the latency profile.

Throughput Changes Too#

The same architecture also changes where the system spends its resources.

Instead of:

we can have:

The workloads are now physically separated.

This implementation measured:

Legacy path:        180 req/s
Read path:        2,400 req/s

an observed 13.3× increase in throughput for that specific read workload.

Again, these are implementation measurements rather than universal characteristics of CQRS or read replicas.

The Cost of Isolation#

There is no free architecture.

We have now introduced:

  • CDC infrastructure,

  • event contracts,

  • message brokers,

  • idempotent consumers,

  • read models,

  • replication,

  • and additional operational monitoring.

That is more software.

But we also changed the failure boundaries.

The old architecture looked like:

The new architecture looks like:

A failure in an AI query does not need to become a failure in checkout.

A slow analytics workload does not need to compete with inventory writes.

A temporarily unavailable consumer can recover from the event stream.

That is the real architectural value.

The Final Failure Model#

At this point, we can describe the system in terms of failure boundaries:

Each boundary has a different responsibility.

Transactional boundary#

MySQL + Outbox

Guarantees that the state mutation and event creation happen together.

Transport boundary#

BinLog + Go Streamer

Moves changes without polling.

Contract boundary#

CloudEvents

Provides a common event envelope and stable metadata.

Temporal boundary#

RabbitMQ / NATS JetStream

Separates producer availability from consumer availability.

Domain boundary#

ACL

Protects the new application from legacy concepts.

Delivery boundary#

Idempotent Consumer

Makes duplicate delivery safe.

Read boundary#

CQRS / Read Model

Separates query workloads from transactional workloads.

That is the complete architecture.

The Golden Rule#

After four articles, the entire design can be reduced to one principle:

Assume events will be delivered more than once, isolate legacy models behind explicit boundaries, and build read paths that do not depend on your transactional core.

The final architecture becomes:

The legacy ERP remains the transactional source of truth.

The event platform becomes the integration boundary.

The consumers own their own domain interpretation.

And AI agents operate against replicated read state instead of reaching into the operational database.

What We Learned#

The four-part journey started with a simple problem:

A cron job was polling a legacy database every 15 seconds.

The final architecture ended somewhere very different:

We did not rewrite the ERP.

We changed the boundaries around it.

That is often the more realistic way to modernize a critical legacy platform.

Not by replacing everything.

By isolating what should remain stable, extracting what needs to evolve, and making failure an explicit part of the design.

Final Takeaway#

The hardest part of distributed systems is rarely the first successful message.

It is the message that arrives:

twice.

At the wrong time.

After a timeout.

After a crash.

After a failover.

After the consumer has already committed the business operation.

Production systems are not defined by what happens when everything works.

They are defined by what happens when everything almost works.

That is why:

At-Least-Once + Idempotency
beats pretending the network can guarantee exactly-once execution.

Anti-Corruption Layers
beat leaking legacy models across new boundaries.

Read Models
beat sending every AI and API query to the transactional core.

And:

Event-driven architecture
beats asking a legacy database the same question every 15 seconds.

The goal is not to make the old system disappear.

The goal is to make it stop being the failure domain for everything around it.