Skip to main content
AI memory systems fail when they let neural networks create entities without validation. The result: duplicates everywhere. “Dave”, “David”, and “Dave Smith” become three different people. Worlds solves this with a clear boundary: neural networks extract and discover, rules define and enforce.

The neural network role

The neural network does two things:
  • Extract information from unstructured data: It reads meetings, documents, and conversations. It identifies entities, relationships, and facts.
  • Provide fuzzy search: It converts text to vector embeddings. This allows semantic similarity search, finding conceptually related information even when keywords don’t match.
The neural network does not decide what entities exist. It does not create new entity types. It does not resolve duplicates.

Search implementations

Worlds ships two search index implementations:
  • RdfjsSearchIndex (in-memory): Keyword-only substring matching. Useful for testing and development. Does not support vector embeddings.
  • SqliteSearchIndex (durable): Full hybrid search with FTS5 keyword search and sqlite-vec vector search. Supports Reciprocal Rank Fusion (RRF) to combine both signals. Requires @worlds/sqlite and an EmbeddingService.

How rules are set

Rules are set through ontology and schema:
  • You define the columns: The ontology specifies what entity types exist (Person, Meeting, Decision) and what predicates connect them (attends, decided, followsUp).
  • The schema enforces consistency: Before any information is written to the graph, the system validates it against the ontology. If the neural network extracts “Dave” and “David” and “Dave Smith”, the system checks the schema for entity resolution rules.
  • SPARQL enforces deterministic retrieval: Once information is in the graph, retrieval is rule-based. The SPARQL engine executes exact graph traversal with no fuzzy matching and no guessing.
  • Write policy governs updates: The model cannot autonomously create arbitrary entities. Writes follow predefined rules. The system validates before committing.

The boundary

Neural networks propose. Rules dispose. The neural network extracts “Dave” from a meeting transcript. The ontology defines that “Dave” is a Person with a givenName. The schema validates that the name follows the expected format. SPARQL retrieves the exact facts about Dave from the graph. If the neural network tries to create a new entity type not in the ontology, the system rejects it. If it tries to write without validation, the system blocks it. The boundary is explicit.

Case study: entity resolution

The entity resolution guide shows this boundary in practice. It covers two patterns:
  • Pattern A: Resolve at retrieval time. Worlds indexes label literals as search aliases, so “Sarah”, “Sarah Chen”, and “S. Chen” resolve to the same subject IRI.
  • Pattern B: Resolve at write time. The pipeline extracts mentions, retrieves candidates with hybrid search, scores with three signals (name similarity, co-occurrence, recency), and decides conservatively.
Both patterns follow the same boundary: neural networks extract and discover, rules validate and enforce.

Runnable examples

The problem: duplicates without rules

Without ontology, the neural network creates separate entities for each mention:
Three entities for one person. The graph fragments.

The solution: ontology prevents duplicates

With ontology, you define that Person entities have names and that names are aliases:
The schema enforces that names are strings on Person entities. The neural network extracts the names, but the ontology defines what a Person is.

Hybrid search: keyword + SPARQL (in-memory)

The in-memory RdfjsSearchIndex does keyword-only substring matching. Search uses two steps:
  • Keyword search discovers the starting point (substring match on literals)
  • SPARQL deterministic retrieval gets exact facts
The keyword search finds the starting point. SPARQL retrieves the exact facts. No guessing.

Hybrid search: neural + SPARQL (SQLite with vector embeddings)

The SqliteSearchIndex supports full hybrid search with vector embeddings. This enables semantic similarity search where natural language queries find conceptually related information.
The neural network finds the starting point through semantic similarity. SPARQL retrieves the exact facts. No guessing. Note: The example above uses Google’s text-embedding-004 through the AI SDK. For fully local embeddings without an API key, the @worlds/sdk repository vendors a TF.js Universal Sentence Encoder service you can copy from examples/tfjs-universal-sentence-encoder/.

AI agent tools: neural network using rules

When an AI agent uses Worlds, it calls tools that enforce the boundary:
The LLM decides what to query. The tools enforce the boundary. The graph provides verified facts.

What you build vs what Worlds provides

Next steps

Try it in the playground

Paste the dataset below into the SPARQL Playground left panel, then run the queries in the right panel.

Dataset

Find all names for a person

Find people by any name variant

Find who works where

Verify a fact

Count people by role