Real-time context layer
Agents act on what is true now, and know how old every fact is.
Typical stack Debezium, Apache Flink, Redis, Postgres, MCP
What good looks like
- Change data capture with ordering and replay guarantees, instead of polling
- Stateful stream processing that handles late, duplicate and out-of-order events
- Serving stores sized for agent latency, with a freshness target for each entity
- Every value returned with its age and source, so an agent can refuse stale data
- Access rules enforced at the tool boundary, not left to the prompt
- Point-in-time replay, so you can audit exactly what an agent saw when it acted
Semantic layer
One definition of revenue, customer and order, used by people and agents alike.
Typical stack Apache Iceberg, Trino, dbt, Spark
What good looks like
- Metrics and entities defined once, as version-controlled code
- Approved joins and grain rules, so no query can double count
- Row- and column-level access rules inherited by every consumer
- One query interface shared by dashboards and agents
- Tests on every definition, run before anything ships
- Query data where it lives when copying it is not worth the cost
Lakehouse and ERP data
The history underneath: cheap to run, easy to replay and trusted by both people and agents.
Typical stack Databricks, Delta Lake, Apache Iceberg, Spark, BigQuery
What good looks like
- Raw, cleaned and modelled layers on open table formats
- Data quality checks and contracts between every layer
- Backfill and replay paths that can rebuild the real-time layer
- ERP extraction that respects source-system load and keeps change history
- Cost governance across compute sizing, storage tiers and job tuning
- Orchestrated pipelines with CI/CD, replacing scheduled scripts