Concepts

How Engram Works

The knowledge graph, node types, and retrieval pipeline — explained.

The knowledge graph

Engram stores facts as a directed acyclic graph (DAG) in a local SQLite database. Each project gets its own isolated graph at ~/.engram/projects/<project>/context.db.

The graph has two primitives:

  • Nodes — individual facts: a decision, a constraint, an implementation detail, an open question, etc.
  • Edges — directed relationships between facts: depends_on, flows_to, relates_to, supersedes.

When you run engram extract, Engram sends your transcript or document to an LLM (Gemini 2.5 Flash by default). The LLM identifies discrete facts and the relationships between them, returning structured JSON. Engram assigns stable UUIDs to each node, deduplicates against existing content (via normalized text hashing), and writes the result to the graph.

Superseded facts are tracked explicitly: if a new decision replaces an older one, the new node carries a supersedes edge pointing at the old node. Both are stored, so the full history is preserved, but retrieval shows only the current state by default.

Projects

Projects are isolated memory stores. Facts from one project never appear in queries from another. Use separate projects for separate codebases, clients, or domains.

# Create projects for two separate codebases
engram init api-backend
engram init mobile-app

All project data lives under ~/.engram/projects/ by default. You can change this with the projects_dir setting in config.yaml.

Extraction

Extraction is the process of turning unstructured text into graph nodes. The extractor sends your text to an LLM with a structured prompt that instructs it to identify facts and categorize them by node type.

The extractor has two modes:

  • Standard (engram extract): single LLM pass over the full document.
  • Verified (engram extract --verify): a second pass focused specifically on secondary details, buried numerics, transition statements, and rationale — facts that the first pass tends to miss.

Targeted passes are also available for specific node types: --decisions, --lessons, --questions, --constraints. Each targeted pass sees the already-extracted nodes to avoid duplication.

Numeric values embedded in fact text are automatically added to the node's tag list so they can be found by value-based queries (e.g. searching "15-minute" will find a node whose fact mentions a 15-minute timeout even if the extractor didn't tag it).

Node types

Engram classifies every fact into one of nine node types. Type determines where the node appears in query output (more important types are shown first) and how it is weighted during retrieval.

Type What it represents Examples
decision Architectural or implementation choices that were consciously made "Chose PostgreSQL over MySQL for JSONB support"
transition Evolution from one approach to another — "moved from X to Y" "Migrated from REST polling to WebSocket push"
constraint Hard requirements, platform restrictions, deadlines, or non-negotiables "Must support Python 3.13; 3.14 breaks sqlite-vec"
implementation How something was built, configured, or wired together "Auth middleware checks JWT in Authorization header, falls back to cookie"
question Open items and unresolved decisions "Should rate limiting live at the API gateway or per-service?"
resolved Previously open questions that have been answered "Decided to use Kong at the gateway layer for rate limiting"
lesson_learned Things that didn't work and why — failure postmortems "Mocked DB tests passed but prod migration failed — integration tests now required"
preference User preferences, style choices, and recurring patterns "Prefer terse responses without trailing summaries"
process Repeatable workflows, team protocols, and step-by-step procedures "Deploy checklist: run tests → tag release → push to staging → smoke test → promote"

Output ordering

Query output groups nodes by type in this priority order: decision → transition → constraint → implementation → resolved → preference → process → question → lesson_learned. Decisions appear first because they are typically the highest-signal facts for an AI editor.

Retrieval

When you run engram query, Engram finds and ranks the facts most relevant to your question. The pipeline has four stages:

1. Keyword extraction

Engram extracts keywords from your query by stripping stop words and splitting hyphenated compounds. For example, "hot-path caching decisions" produces the keywords hot-path, hot, path, caching, decisions. Numeric tokens and special identifiers (e.g. rs256, 1000/min) are preserved exactly so you can search by value.

2. Entry node discovery

Engram searches the graph for nodes whose tags overlap with the extracted keywords. The more keyword matches a node has, the higher it ranks as a starting point for traversal. This is the relevance_scoring strategy.

3. BFS graph traversal

Starting from each entry node, Engram walks the graph up to hops steps (default: 3) in both directions — following outgoing edges forward and incoming edges backward. This pulls in related context: if a constraint node is relevant, you also get the decisions that depend on it.

4. Strategy pipeline

The collected neighborhood is filtered and ranked through a series of strategies applied in order:

Strategy Default What it does
superseded_pruning on Drops nodes that have been superseded by a newer node. The old version is preserved in the graph but hidden from results.
confidence_threshold 0.0 (off) Filters out nodes below a minimum confidence score. Set to e.g. 0.6 to exclude low-certainty extractions.
recency_decay off Multiplies each node's score by 2^(-age_days / half_life_days). Older facts rank lower. Half-life is configurable (default: 30 days).
top_k 10 Returns only the top K nodes by score after all other strategies have run.
token_budget 0 (unlimited) Greedily packs nodes into output until the estimated token count is reached (~4 chars/token). Useful for fitting output into a specific context window slot.

Strategies are independently toggleable. You can override them per-query with --enable <strategy> / --disable <strategy>, or permanently in config.yaml.

Temporal queries

Every node carries a timestamp (occurred_at). When you use --at-time, the temporal_valid_at strategy is automatically enabled and filters to only nodes that were valid at the specified time. This lets you ask what the project state looked like at any historical point.

Deduplication

When a fact is extracted that already exists in the graph (determined by a normalized text hash), the duplicate is silently dropped. This means re-running extraction on overlapping documents is safe — you won't accumulate copies of the same fact.

For semantic near-duplicates (similar wording, different text), use engram dedup to find and merge them. The dedup command compares node embeddings and presents candidates above a configurable cosine-similarity threshold.

Pinned nodes

Nodes can be pinned with engram pin <project> <node-id>. Pinned nodes are always included in query output regardless of score. Use pinning for facts that must always be in context — project conventions, invariants, or persona definitions.

Next steps