The Data Agent Stack - Part 4: Meaning Lives in Code
Why schemas describe shape, but pipeline code explains meaning
A user asks for active users.
The agent finds a table with user_id, event_time, and platform.
The SQL runs. The chart looks reasonable.
But the table excludes logged-out traffic. It deduplicates at session grain, not user-day grain. Yesterday’s partition exists, but the upstream mobile-events job has not finished.
The schema was correct.
The answer was wrong.
A table’s schema tells you what columns exist. Pipeline code explains how the table was produced, what assumptions it encodes, and when it should be trusted.
This post is not about teaching a model to read every file in your repo at query time.
It is about turning production code into a durable table-understanding layer: extracted, refreshed, access-controlled, linked to provenance, and retrieved before the agent writes SQL.
OpenAI’s public writeup on its in-house data agent makes this pattern concrete. The post describes multiple context layers: table usage, human annotations, Codex enrichment, institutional knowledge, memory, and runtime context. It describes Codex enrichment as deriving code-level table definitions, including purpose, grain, primary keys, downstream usage, alternate table options, and freshness.
My read is that the reusable pattern is not “copy OpenAI’s implementation.”
The reusable pattern is this:
A production data agent needs a maintained table context card. Pipeline code is one of the highest-authority inputs into that card because it reveals executable behavior.
Schemas describe shape.
Trusted query history describes usage.
Owner annotations describe intent.
Pipeline code explains behavior.
The data agent needs all of them, but it should not treat them as equal.
The schema tells you shape, not meaning
Schemas matter.
A schema tells the agent column names, data types, nullable fields, partition columns, and the rough shape of a query.
That is useful. If a table has event_time, the agent knows it can filter by time. If it has country, it can group by geography. If it has user_id, it may be able to join to a user dimension.
But schemas do not answer the harder questions.
What is the grain?
Is one row one event, one session, one user per day, one account, one subscription, or one deduplicated business entity?
Which rows were dropped?
Does the table include logged-out users, bot traffic, internal test accounts, deleted records, refunded transactions, backfilled partitions, or late-arriving events?
Which upstream source is canonical?
Is this table safe for reporting, or is it a staging table with a friendly name?
How fresh is it?
A column type does not answer that.
Spotify’s data assistant post gives the same lesson from a different angle. Spotify explains that schemas alone do not capture business-specific meaning. An INT64 column does not tell you that values below 100 are legacy test data, or what a team means by “active user.”
That is the first failure mode for data agents.
The agent sees the shape.
It misses the contract.
In normal analytics work, humans often carry the contract in their heads. A senior analyst knows which table includes test accounts. A data engineer knows which backfill is incomplete. A product data scientist knows which active-user definition belongs to which surface.
A data agent does not know that unless the platform makes it legible.
That is why semantic layers, catalog context, lineage, owner annotations, institutional knowledge, and code-enriched context matter.
A semantic layer can define entities, dimensions, measures, and metrics. dbt’s semantic model docs, for example, describe semantic models as the foundation for data definition in MetricFlow and define entity types such as primary, unique, foreign, and natural keys.
But a semantic layer does not automatically explain every production table.
For that, the agent often needs the code that produced the data.
Query history helps, but it can teach old habits
Historical queries are useful.
They show joins people actually use.
They show filters that repeat across dashboards.
They show common aggregations.
They show which tables analysts reach for when answering similar questions.
That is why query history appears in serious data-agent systems. OpenAI’s post describes table usage as schema metadata, lineage, and historical queries that help the agent understand how tables are typically joined. Pinterest’s analytics-agent post describes extracting join keys, filters, aggregation logic, usage signals, freshness, documentation quality, and table tiers to rank results by trust, not just similarity.
But query history is not truth.
It is behavior.
Behavior can be good, stale, experimental, or wrong.
Someone may have written a query for a one-off incident. Someone may have copied an old dashboard. Someone may have used the wrong table because the right one did not exist yet. Someone may have added a launch-specific filter and never removed it.
Spotify has the cleanest public example here. During curation, Spotify generated question-SQL pairs from real query history and asked domain experts to review them. Only 12.5 percent were accepted. The rest included ad hoc exploration, debugging, one-off answers, wrong-table usage, and technically correct queries that taught the wrong pattern.
That is the second failure mode.
The agent does not only learn from your best analysts.
It learns from the residue of your warehouse.
Query history is useful because it shows behavior. It is dangerous because behavior is not authority.
This does not mean query history is bad.
It means query history needs authority.
A production dashboard query should carry more weight than a notebook experiment.
A repeated query from a domain team should carry more weight than a one-off debug query.
A curated question-SQL pair should carry more weight than a generated pair that no human reviewed.
A practical source hierarchy looks like this:
When these sources disagree, the agent should not average them into a confident answer.
It should rank them, surface the conflict, or ask for clarification.
A query without provenance is just an opinion with SQL attached.
Pipeline code is where the hidden contract lives
Pipeline code is not just implementation.
For a data agent, pipeline code is executable documentation.
It shows how a table was created, not just what columns it has.
That matters because the code often contains the hidden contract:
table grain
primary keys
deduplication rules
filters and exclusions
update cadence
freshness expectations
upstream dependencies
downstream consumers
fallback tables
implemented business rules
scope caveats
OpenAI’s public post describes Codex enrichment as deriving code-level table definitions. It says this layer helps the agent understand what a table contains, how it is derived, uniqueness, update frequency, data scope, granularity, usage in Spark or Python, and differences between similar-looking tables. It also says this context is refreshed automatically.
This changes the question the agent should ask.
Not:
Which table has the right columns?
But:
Which table has the right meaning for this question?
Those are different questions.
Consider two user activity tables.
Both have user_id.
Both have event_time.
Both have platform.
One table includes logged-out traffic and deduplicates events at session grain.
The other includes only authenticated first-party usage and deduplicates at user-day grain.
A schema retriever may rank both highly.
A historical query may show both used for “activity.”
The pipeline tells you the difference.
Or take revenue.
One table stores gross bookings. Another stores net revenue. Another excludes refunds. Another includes credits but excludes taxes. Another filters internal test accounts. Another refreshes after finance reconciliation.
A model can write syntactically valid SQL against any of them.
The question is whether the table matches the metric contract.
Or take late-arriving events.
The schema has event_date.
The table is partitioned by date.
Yesterday’s partition exists.
But the pipeline expects late mobile events until 10 AM. At 8 AM, the table exists but the answer is not final.
That is not a SQL problem.
That is a freshness and orchestration problem.
Grab’s engineering post shows this in a practical investigation workflow. Their system uses a Code Search Agent to trace column transformations through a codebase, follow table lineage across transformation steps, generate plain-English explanations of transformation logic, and highlight divergence from documentation or stakeholder expectations.
Sometimes the answer to “what does this value mean?” is not in the schema.
It is in the transform.
But code is not the whole truth.
Pipeline code is the highest-authority source for executable behavior. It shows what the system actually does when it builds the table.
Business meaning may live in a metric contract.
Caveats may live in owner annotations.
Freshness may need orchestration state.
Usage may come from trusted dashboards.
The agent should reconcile these sources, not collapse them into one summary.
Code-enriched context is not a repo dump
There is a tempting design:
Give the model access to the whole repo and let it figure it out.
That sounds powerful.
It usually creates a new problem.
Large repos are noisy. They contain old code, tests, migrations, generated files, dead paths, temporary notebooks, deprecated jobs, and comments that no longer match executable logic.
A data agent does not need every line of pipeline code in the prompt.
It needs table-centered facts extracted from code.
The useful artifact is the table context card.
That card should combine:
schema metadata
trusted query usage
owner annotations
semantic definitions
lineage
orchestration state
pipeline-derived facts
runtime checks
provenance
confidence
permission scope
The context card should not say only:
This table contains user events.
That is too weak.
It should answer:
What is the table for?
What is the row grain?
What is the primary key or uniqueness assumption?
What upstream sources feed it?
Which filters or exclusions are applied?
How is deduplication handled?
How often is it updated?
When is a partition considered final?
Which dashboards, notebooks, models, or jobs consume it?
Which similar tables should be used instead for other questions?
Who owns the table?
Where did each fact come from?
When was the context last refreshed?
Who is allowed to retrieve this context?
This is where Code Wiki, DeepWiki, and similar code-understanding systems are useful as a pattern.
Google’s Code Wiki maintains a continuously updated structured wiki for code repositories, regenerates docs after code changes, links explanations back to source files and definitions, and can generate architecture, class, and sequence diagrams. Devin’s DeepWiki indexes repositories and produces architecture diagrams, documentation, source links, summaries, and code-grounded Q&A.
That is useful infrastructure.
But a data agent needs a narrower artifact.
A code wiki explains the repo.
A data agent needs to explain the table.
The table is the retrieval unit. The table is the access-controlled asset. The table is what the SQL will touch. The table is what the answer depends on.
My read is that the best implementation borrows from code-wiki systems, but specializes the output for data-agent reasoning:
not “how does this repo work?”
but “what does this table mean, how was it built, and when is it safe to use?”
A code wiki explains the repo. A data agent needs to explain the table.
That is the difference between code search and code-enriched data meaning.
Freshness is part of meaning
Freshness is not a side field.
Freshness is part of what a table means.
A daily table and an hourly table are not interchangeable.
A partition that exists and a partition that is complete are not the same thing.
A source that has arrived and a downstream model that has finished are not the same thing.
dbt’s freshness docs define freshness as the acceptable amount of time between the most recent record and now for a table to be considered fresh, with warning and error thresholds for stale sources. Airflow’s DAG docs describe schedules, tasks, dependencies, callbacks, and operational details as part of how workflows execute.
For a data agent, these details matter in ordinary questions:
What happened yesterday?
How did revenue look after launch?
Did active users drop this morning?
Is this incident affecting signups?
The right answer may depend on whether:
the upstream DAG completed
the latest partition is final
late-arriving events are still expected
a backfill is running
a source table is stale
a downstream model skipped a run
a partial refresh succeeded
the table moved from one pipeline to another
This is where code-derived context and runtime context need to work together.
Offline enrichment can say:
This table is produced daily after the events aggregation DAG completes. It is usually safe for yesterday after 10 AM.
Runtime validation can check:
Did the DAG finish today?
Is yesterday’s partition complete?
Did row counts look normal?
Did the freshness assertion pass?
Without runtime validation, code-derived context becomes stale documentation.
Without code-derived context, runtime checks do not know what to check.
The code explains the intended contract.
Runtime state verifies whether the contract holds right now.
At 3:00 AM, the operator does not need a plausible paragraph. They need to know which code version, which context card, which freshness check, which query, and which runtime state produced the answer.
What builders should take from this
Code-enriched context should be built like a metadata product.
It needs a schema.
It needs owners.
It needs refresh triggers.
It needs provenance.
It needs permission scope.
It needs confidence.
It needs tests.
The consumer is the data agent, but the artifact should be useful to humans too.
The point is not to copy any one catalog model. The point is to treat context as governed infrastructure, not prompt decoration.
DataHub’s metadata model is useful background because it treats metadata as typed entities, aspects, and relationships in a graph. OpenLineage is useful because it models how datasets come into being through jobs, run events, dataset events, and lineage facets. Databricks Unity Catalog lineage is another useful reference point because it connects tables, jobs, notebooks, queries, dashboards, upstream sources, downstream consumers, and permissions.
The implementation will vary by stack.
The invariant should not.
The agent should not reason over anonymous context.
It should reason over maintained context with source, freshness, authority, and permission boundaries.
A data agent does not become reliable because it writes SQL. It becomes reliable because the platform makes meaning legible before the model reasons over it.
Failure modes
1. Schema-accurate, semantically wrong table
The table has user_id, event_time, and platform, but excludes logged-out traffic.
The SQL is valid.
The population is wrong.
2. Grain mismatch creates fanout
The agent assumes one row per user-day.
The table is event-grain.
A join to account data multiplies rows and inflates the metric.
3. Freshness intent does not match runtime state
The context card says the table is usually ready by 10 AM.
Today’s upstream DAG failed.
The agent should caveat, retry, or use an alternate table.
4. Query history teaches old logic
The agent copies a repeated query from a deprecated dashboard after the pipeline migrated to a v2 table.
The query used to be reasonable.
Now it is a regression.
5. Similar tables cannot be disambiguated
Two tables share names, columns, and partitions.
One is canonical for reporting. One is raw ingestion. One excludes a product surface.
Schema similarity hides meaning.
6. Pipeline-derived context is stale
The card says refunds are excluded.
Last week’s transform changed the logic.
The table description now misleads the agent.
7. Extracted facts have no provenance
The card says “use this table for paid subscribers,” but does not identify whether that came from code, a comment, a dashboard, or an LLM summary.
Without provenance, the agent cannot rank the claim.
8. Comments disagree with code
The comment says test accounts are excluded.
The filter was removed.
The agent trusts the comment because it is readable.
9. Static extraction misses dynamic logic
The relevant SQL is generated through macros, runtime config, template expansion, or framework code.
The extractor summarizes the wrapper, not the executed query.
10. Context crosses permission boundaries
A code-derived context card reveals sensitive downstream consumers, experiment names, or restricted datasets to a user who cannot access them.
That is not a retrieval bug.
That is a governance bug.
Builder checklist
Create a table context card schema. Include purpose, grain, keys, filters, exclusions, freshness, upstreams, downstreams, alternate tables, owner, provenance, confidence, and permission scope.
Extract executable behavior from code. Pull facts from dbt models, Spark jobs, Airflow DAGs, Python transforms, SQL files, notebooks, macros, and generated artifacts where possible.
Separate business definition from executable behavior. Metric contracts and owner docs define intended meaning. Pipeline code shows implementation. The agent needs both.
Refresh on change. Trigger enrichment when code, schema, DAGs, table ownership, lineage, or deprecation status changes.
Attach provenance to every extracted fact. Store source file, commit or version, extraction timestamp, owner, and confidence.
Validate before answering. Check schema, partitions, freshness, row counts, distinct values, orchestration state, and data quality signals when needed.
Test context changes. Run golden questions when a table context card or extraction rule changes. GitHub’s Qubot post is a useful public example of evaluating context-layer changes before they ship.
Track source authority and fail visibly on conflict. Rank semantic layer, owner annotations, pipeline code, lineage, trusted dashboards, and raw query history differently. If they disagree, surface the conflict instead of silently choosing one.
Recap
A data agent does not need every line of pipeline code in the prompt.
It needs the meaning that code encodes.
That meaning should be extracted, refreshed, ranked, access-controlled, and made available before the model writes SQL.
Schemas describe shape.
Trusted query history describes usage.
Owner annotations describe intent.
Pipeline code explains behavior.
The table context card is where those signals meet.
That is the difference between a model that can write SQL and a data agent that can answer with evidence.
What comes next
Part 5 moves from context to action: Tools, Query Execution, and the Analyst Loop.
Once the agent has the right context, the next question is what it can do with it.
Which tools can it call?
How does it run SQL safely?
When should it ask a clarifying question?
When should it stop, retry, redirect, or escalate?
And why do fewer, clearer tools often beat more tools?
The Data Agent Stack
References
OpenAI, “Inside OpenAI’s in-house data agent”, primary public case study for context layers, Codex enrichment, runtime context, permissions, evals, and the “Meaning Lives in Code” lesson.
Spotify Engineering, “Encoding Your Domain Expert”, strong source for domain-owned context, query-history noise, curated examples, and context health.
GitHub Blog, “How we built an internal data analytics agent”, useful public example of bronze/silver/gold context, ETL-enriched context, federated contribution, and evals for context-layer changes.
Pinterest Engineering, “Unified Context-Intent Embeddings for Scalable Text-to-SQL”, useful for query-history normalization, governance-aware ranking, source hierarchy, validation, and conflict handling across context sources.
Google Developers Blog, “Introducing Code Wiki” and Devin Docs, “DeepWiki”, useful examples of codebase indexing, source-linked explanations, architecture diagrams, and code-grounded Q&A.
dbt Semantic Models, dbt Freshness, OpenLineage Object Model, and DataHub Metadata Model, useful references for semantic entities, freshness checks, lineage, typed metadata, ownership, and graph-based context.