An ontology that learns to be useful

A field name is not a concept

Suppose an analyst asks: “Which cards are banned in the Modern format?” The data is in a database, but its fields are cards.uuid, legalities.uuid, legalities.format, and legalities.status. A SQL tool can read these fields; it cannot tell the agent what “banned” means, that the status depends on a format, or which keys to join.

That is the agent–data gap. The raw source holds facts, while the agent reasons in business concepts. A prompt containing every table, document and metric is not a scalable fix: much of it is irrelevant to the current request, consumes context, and can distract the model. A static semantic glossary helps, but may lack field-level grounding and does not adapt when agents repeatedly stumble.

Three layers, three different jobs

The paper represents version t of its ontology as L_t = (S_t, Γ_t, R_t), where S is content, Γ is schema and R is the runtime tool interface. Distinguishing them matters: “the status value is undocumented” is a content defect; “browse never surfaces the status term” is a tool defect; “the object model cannot represent format scope” is a schema defect.

At runtime, browse(q, kind, n) finds the top n matching semantic objects. resolve(ids, context) expands selected records and their links. The session manifest is a compact map of sources and tool use; according to the paper, it is the only ontology content injected initially into the prompt. The authors encapsulate this interface as an MCP server.

Ground first. Revise from failures. Gate every change.

Here V is the validation set, m the deployed LLM backbone and τ an acceptance margin. This is paired selection, not proof that the patch is universally better: a small validation set may still overfit. The paper evolves separate versions for different backbones.

From “banned” to an executable answer

This example follows the paper’s card-legality case, with invented toy rows for clarity. The initial graph already knows Card and Legality and where their columns live, but it does not say how to interpret status or that format must be specified. A query for banned cards might erroneously include cards banned in a different format.

The proposed content edit adds a Legality Status Code term, maps it to legalities.status, stores observed values as evidence, and attaches the constraint status = 'Banned' AND format = target_format. Then the agent can browse the term, resolve its mapping/rule/evidence, and use its existing SQL tool to execute:

SELECT DISTINCT c.name
FROM cards AS c
JOIN legalities AS l ON c.uuid = l.uuid
WHERE l.status = 'Banned'
  AND l.format = :target_format;

For target Modern, the correct result is Ember Wisp, not Tidal Sage or Stone Finch. The ontology does not hold the authoritative answer; it helps the agent compose the right query against the source.

Try the semantic interface

Switch the target format, disable graph objects, compare manifest visibility, then test a proposed content patch. This is an illustrative deterministic simulation, not the paper’s software or a live LLM. Its “agent” has deliberately limited behavior so you can see what each layer contributes.

What changed in the experiments?

The authors compare a ReAct agent with no ontology, a ReAct agent with a builder-produced semantic layer pasted into the prompt, and EvoOntology with interactive tools and evolution. Benchmarks cover multi-source 10-K research (DDR-Bench), CSV business insights (InsightBench), and text-to-SQL (BIRD). Both folds are evaluated: adapt with 70% of one fold, gate with its remaining 30%, freeze, test on the other fold, then swap.

The BIRD +8.8 is the paper’s stated four-backbone initial/evolved analysis (+5.1 then +3.7), not the six-backbone main-table mean (+7.4). The paper reports six backbones in its main tables and four in several analyses; its abstract says four, so read the specific experimental section when comparing numbers. Percentages here are percentage-point differences, not relative-percent gains.

The static pasted semantic layer is not uniformly helpful: on DDR-Bench Claude-Sonnet-5, its trajectory-wise score drops from 72.5% to 57.5%, while interactive EvoOntology scores 81.3%. This comparison changes both how the layer is delivered and whether it is evolved, so it does not by itself isolate the effect of MCP tools from the effect of evolution. The initial-vs-evolved analysis helps isolate the second piece.

Cost is a trajectory story, not a per-call story. Average DDR-Bench input tokens per turn rise from 3.2K to 4.6K, but turns per task fall from 14.6 to 8.4; reported total tokens fall from 52.6K to 42.0K. That is roughly a 20% reduction in this setting, not a general guarantee of lower latency or infrastructure cost.

A practical implementation sketch

// Minimal illustrative records (not the authors' implementation)
term = { id: 'legality_status', label: 'Legality Status Code' }
mapping = { term: 'legality_status', field: 'legalities.status',
            join: 'cards.uuid = legalities.uuid' }
constraint = { term: 'legality_status',
               predicate: "status = 'Banned' AND format = :target_format" }
evidence = { term: 'legality_status', probe: 'SELECT DISTINCT status FROM legalities',
             observed: ['Banned', 'Legal'] }

browse('banned in Modern', 'term', 3)  // → [legality_status, ...]
resolve(['legality_status'], {format:'Modern'})
// → term + mapping + constraint + evidence

For a production context graph, add access-control propagation from underlying sources, evidence freshness and source lineage, version pinning, safe SQL generation, audit logs, and rollback. These are engineering recommendations, not measured features of the paper. Keep private rows out of a shared manifest; authenticate browse and resolve at the object and source level.

What this paper does—and does not—establish

Source: EvoOntology: A Self-Evolving Ontology Layer for Data Agents · original paper (PDF). Numbers are taken from its tables and appendices; toy data and architecture interpretation are original explanations in this report.