Rebuilding stateful CDC pipelines without replaying the world

Posted in Brex Engineering Blog

Part 1 of a Brex engineering series on rebuilding stateful Flink CDC pipelines with federated snapshots.

Making CDC snapshot merges correct

Posted in Brex Engineering Blog

Part 2 of the series, covering the keyed-state merge rules that keep live CDC mutations ahead of stale snapshot rows.

Operating hundreds of stateful Flink pipelines

Posted in Brex Engineering Blog

Part 3 of the series, on the deployment, validation, rollback, and operating model behind Brex's stateful Flink platform.

Data continues to be the moat

AI is making it easier to render software surfaces. The harder thing to replace is the data a product stores, protects, and makes useful.

Reviewing Anthropic's AI analytics stack

The headline is Claude doing analytics. The actual story is the unglamorous work of making the data model usable.

What a production eval system actually looks like

A production eval system is not a notebook. It is an operational feedback loop, and the labels are usually the hard part.

Stop building AI features

Some AI product work gets commoditized quickly. The durable work tends to live lower in the stack: data, permissions, workflows, and integrations.

Data catalogs are only useful if they match real workflows

Metadata catalogs fail when they become polished inventories. The useful ones fit the decisions people are already trying to make.

The semantic layer is execution infrastructure

Agents turned semantic layers from BI cleanup into runtime infrastructure. Many AI reliability problems are really modeling problems.

Zero-downtime Flink upgrades

Savepoints are great for recovery. They are a fragile deployment primitive once stateful jobs start evolving quickly.

Transitioning from PM to SWE

Moving from product to engineering gave me a better angle on systems work. It also forced me to unlearn some habits I did not know I had.

Domain-driven pipelines

One perfect data pipeline sounds clean until every domain needs it to mean something different.

DI and the Cake Pattern

The cake pattern is clever Scala. Constructor injection is usually the better default.