The pipeline
- Chunking: Break documents into semantic segments.
- Embedding: Generate vector representations for each segment.
- Extraction: Use LLMs to identify items and relationships within the text.
- Linking: Store segments as
chunksand relationships astripleswithin the world memory.
Benefits of graph-backed documents
- Contextual continuity: Follow relationships between different parts of a document or across multiple documents.
- Improved recall: Retrieve specific document segments based on their logical role.
- Verification: Cross-reference extracted facts against the original document chunks to ensure accuracy.