Lex

Episodic Memory & Architectural Policy for AI Agents

Stop losing context. Start building agents that remember.

Quick Start · Documentation · Examples · API Reference · Contributing

📖 What is Lex?

Lex is a TypeScript framework that gives AI agents episodic memory and architectural awareness. It solves the fundamental problem of context loss in long-running development workflows.

The Problem

You're working with an AI coding assistant on a complex feature. You stop for the day. When you return:

The assistant has no memory of what you were doing
It can't recall why you made certain architectural decisions
It doesn't know which modules are safe to modify
You spend 30 minutes re-explaining context every session

The Solution

Lex provides three capabilities:

📸 Episodic Memory (Frames) — Capture work snapshots with context, blockers, and next actions
🗺️ Spatial Memory (Atlas) — Navigate module dependencies without overwhelming token budgets
🛡️ Architectural Policy — Enforce boundaries, permissions, and deprecation patterns in CI

Result: Your AI assistant recalls exactly where you left off, understands your architecture, and respects your constraints.

🎯 Core Capabilities

🧠 Frames: Work Session Memory

Capture meaningful moments in your development workflow:

lex remember \
  --reference-point "Implementing user authentication" \
  --summary "Added JWT validation to API middleware" \
  --next "Wire up password reset flow" \
  --modules "services/auth,api/middleware" \
  --blockers "Need PermissionService access - forbidden edge in policy" \
  --jira "AUTH-123"

Later, instantly recall:

lex recall "authentication"
# Returns: Your exact context, blockers, next action, and relevant module neighborhood

Learn more about Frames →

🗺️ Atlas Frames: Architectural Context

When you recall a Frame, Lex doesn't dump your entire codebase into context. Instead, it provides an Atlas Frame: the modules you touched plus their immediate neighborhood (dependencies, dependents, permissions).

This "fold radius" approach gives AI assistants exactly the architectural context they need—nothing more, nothing less.

Token efficiency: 10-module project → ~500 tokens (not 50,000)

Learn more about Atlas →

🛡️ Policy Enforcement

Define architectural boundaries as code:

{
  "modules": {
    "ui/components": {
      "owns": ["src/ui/components/**"],
      "mayCall": ["services/auth", "ui/shared"],
      "forbidden": [
        {
          "target": "database/queries",
          "reason": "UI must not access database directly. Use API layer."
        }
      ]
    }
  }
}

Enforce in CI:

lex check merged-facts.json
# ✖ Violation: ui/components → database/queries (forbidden edge)
#   Reason: UI must not access database directly. Use API layer.

Learn more about Policy →

📝 Instructions Generation

Maintain a single source of truth for AI assistant guidance:

# Your canonical instructions live in one place
.smartergpt/instructions/lex.md

# Project them to host-specific files
lex instructions generate
# Creates: .github/copilot-instructions.md, .cursorrules

Benefits:

Single source → Multiple hosts (Copilot, Cursor, etc.)
Safe updates via marker system (human content preserved)
Deterministic output (same input = same output)

Learn more about Instructions →

🚀 Quick Start

Installation

# Install globally (recommended)
npm install -g @smartergpt/lex

# Or locally in your project
npm install @smartergpt/lex

Requires Node.js 20+ (tested through Node.js 22, see .nvmrc)

Lex 2.0.0 is AX-native with structured output (--json), recoverable errors (AXError), and Frame Schema v3 for orchestrator integration. All commands provide both human-readable and machine-parseable output.

Initialize

# Zero-to-value initialization (new in v2.1.0!)
npx @smartergpt/lex init --yes
# Auto-detects project type (Node.js, Python, Rust, Go, etc.)
# Creates:
#   .smartergpt/ - Workspace with prompts, policy, and instructions
#   .github/copilot-instructions.md - IDE instructions with LEX markers
#   .cursorrules - Cursor IDE instructions (if Cursor detected)
#   lex.yaml - Configuration with sensible defaults
#   .smartergpt/lex/memory.db - SQLite database (initialized on first use)
# Shows MCP server configuration guidance
# Idempotent: safe to run multiple times

# Interactive mode (prompts for first Frame)
lex init --interactive

# Generate seed policy from directory structure
lex init --policy
# Scans src/ for TypeScript/JavaScript modules
# Generates .smartergpt/lex/lexmap.policy.json with discovered modules
# Example: src/memory/store/ → memory/store module with src/memory/store/** match pattern

# Force reinitialize (overwrite existing files)
lex init --force

What happens during init:

🔍 Project Detection — Identifies Node.js, Python, Rust, Go, Java, .NET, Ruby projects and frameworks
📝 IDE Setup — Creates .github/copilot-instructions.md and .cursorrules (if Cursor detected) with LEX markers for safe updates
⚙️ Configuration — Generates lex.yaml with defaults based on your project
💾 Database — Initializes SQLite database at .smartergpt/lex/memory.db
📡 MCP Guidance — Shows configuration for VS Code MCP server integration

Capture Your First Frame

lex remember \
  --reference-point "Refactoring payment processing" \
  --summary "Extracted validation logic to PaymentValidator" \
  --next "Add unit tests for edge cases" \
  --modules "services/payment"

Recall Later

lex recall "payment"
# Shows your context, blockers, and architectural neighborhood

Database Maintenance

Keep your memory database optimized and backed up:

# Create a timestamped backup (memory-20251123.sqlite)
lex db backup --rotate 7
# Keeps last 7 backups, stored in .smartergpt/lex/backups/

# Optimize database (rebuild and compact)
lex db vacuum

# Set backup retention via environment variable
export LEX_BACKUP_RETENTION=14  # Keep 14 most recent backups

NDJSON Logging: Lex automatically logs operations to .smartergpt/lex/logs/lex.log.ndjson with structured fields:

timestamp, level, message, module, operation, duration_ms, metadata, error
Log files rotate automatically at 100MB
Logs are silent in test mode unless LEX_LOG_NDJSON=1

That's it! You now have persistent memory for your AI workflows.

Full Quick Start Guide →

MCP Server Integration

Use Lex with any MCP-compatible AI assistant (VS Code, Claude Desktop, etc.):

# Install the MCP wrapper
npm install -g @smartergpt/lex-mcp

# Or run directly
npx @smartergpt/lex-mcp

VS Code Configuration (.vscode/mcp.json):

{
  "servers": {
    "lex": {
      "command": "npx",
      "args": ["@smartergpt/lex-mcp"],
      "env": {
        "LEX_WORKSPACE_ROOT": "${workspaceFolder}"
      }
    }
  }
}

Claude Desktop Configuration:

{
  "mcpServers": {
    "lex": {
      "command": "npx",
      "args": ["@smartergpt/lex-mcp"]
    }
  }
}

The MCP server exposes 11 tools for episodic memory, policy validation, and architectural analysis:

frame_create, frame_search, frame_get, frame_list, frame_validate
policy_check, timeline_show, atlas_analyze
system_introspect, help, hints_get

MCP Server Documentation →

💡 Use Cases

👨‍💻 For Developers

Preserve context across work sessions with AI coding assistants
Document architectural decisions with searchable, timestamped snapshots
Enforce boundaries via CI to prevent policy violations

🤖 For AI Agents

Recall previous work using natural language search
Navigate large codebases with token-efficient Atlas Frames
Respect constraints by checking policy before suggesting changes

👥 For Teams

Onboard new members by showing architectural history
Track technical debt with kill patterns and forbidden edges
Maintain consistency across multi-module projects

📸 See It In Action

Want to see how Lex is used in practice? Check out our dogfooding examples — real Frames from building Lex with Lex.

See how we:

Use lex recall to resume work after breaks
Capture context with lex remember at key checkpoints
Handle blockers and escalations in real development
Track complexity and turn costs for governance

View Dogfood Examples →

🏗️ Architecture

Lex is built on three pillars:

┌─────────────────────────────────────────────────────────┐
│                    Lex Framework                        │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  📸 Memory Layer (lex/memory)                          │
│  ├─ Frame storage (SQLite)                             │
│  ├─ Search & recall                                     │
│  └─ MCP server integration                             │
│                                                         │
│  🗺️ Atlas Layer (lex/shared/atlas)                     │
│  ├─ Module dependency graphs                           │
│  ├─ Fold radius computation                            │
│  └─ Token-aware context generation                     │
│                                                         │
│  🛡️ Policy Layer (lex/policy)                          │
│  ├─ Boundary definitions                               │
│  ├─ Multi-language scanners                            │
│  └─ CI enforcement                                      │
│                                                         │
└─────────────────────────────────────────────────────────┘

Architecture Details →

📚 Documentation

Getting Started

Quick Start Guide — Get up and running in 5 minutes
Installation & Setup — Detailed installation guide
Core Concepts — Understanding Frames, Atlas, and Policy

Guides

Mind Palace Guide — Using Frames for episodic memory
Code Atlas Guide — Spatial memory and architectural context
Policy Enforcement — Setting up architectural boundaries
Instructions Generation — Sync AI instructions across IDEs
CLI Reference — Command-line usage and output modes
MCP Integration — Using Lex with Model Context Protocol
OAuth2/JWT Authentication — Multi-user authentication setup

Security

Security Policy — Security posture and best practices
OAuth2/JWT Guide — Authentication and user isolation

Advanced

Architecture Loop — How Frames, Atlas, and Policy interact
API Reference — TypeScript API documentation
Limitations — Known constraints and future work
FAQ — Common questions

Development

Contributing Guide — How to contribute
Release Process — Versioning and publishing
ADRs — Architectural decision records

🔧 API Reference

Lex provides multiple entry points for different use cases:

Core API

import { saveFrame, searchFrames, getDb, closeDb } from '@smartergpt/lex';

const db = getDb(); // Uses .smartergpt/lex/memory.db

await saveFrame(db, {
  referencePoint: 'authentication flow',
  summaryCaption: 'Added password validation',
  statusSnapshot: { nextAction: 'Wire up permission check' },
  moduleScope: ['services/auth', 'services/password'],
  branch: 'feature/auth',
  jira: 'AUTH-123'
});

const results = await searchFrames(db, { referencePoint: 'authentication' });
closeDb(db);

Subpath Exports

Import	Purpose	Documentation
`@smartergpt/lex`	Core API + store operations	API Usage
`@smartergpt/lex/cli`	Programmatic CLI access	CLI Output
`@smartergpt/lex/cli-output`	CLI JSON utilities	CLI Output
`@smartergpt/lex/store`	Direct database operations	Store Contracts
`@smartergpt/lex/types`	All shared types	API Usage
`@smartergpt/lex/errors`	AXError schema and utilities (v2.0+)	AX Contract
`@smartergpt/lex/policy`	Policy loading & validation	API Usage
`@smartergpt/lex/atlas`	Atlas Frame generation	Architecture
`@smartergpt/lex/atlas/code-unit`	Code unit schemas	Atlas
`@smartergpt/lex/atlas/schemas`	Atlas schemas	Atlas
`@smartergpt/lex/aliases`	Module alias resolution	Aliases
`@smartergpt/lex/module-ids`	Module ID validation	API Usage
`@smartergpt/lex/memory`	Frame payload validation	API Usage
`@smartergpt/lex/logger`	NDJSON logging	API Usage
`@smartergpt/lex/lexsona`	Behavioral memory socket (v2.0+)	Control Stack
`@smartergpt/lex/prompts`	Template system	Canon Architecture

Full API Documentation →

🎯 Project Status

Current Version: 2.1.1 (Changelog)

🚀 2.0.0 — AX-Native Release

Lex 2.0.0 is the first stable release with AX (Agent eXperience) as a first-class design principle. This release introduces structured output, recoverable errors, and Frame Schema v3 for AI agent integration.

Ready for:

✅ Personal projects and local dev tools
✅ Private MCP servers
✅ CI/CD policy enforcement
✅ Multi-user deployments with OAuth2/JWT
✅ Encrypted databases with SQLCipher
✅ LexRunner and other orchestrator integrations

2.0.0 Highlights:

AX Guarantees (v0.1) — Structured output, recoverable errors, reliable memory/recall (AX Contract)
Frame Schema v3 — Runner fields (runId, planHash, toolCalls) for orchestration (Schema Docs)
AXError Schema — Structured errors with code, message, context, and nextActions[] for programmatic recovery
CLI JSON Output — lex remember --json and lex timeline --json with machine-parseable event streams
Instructions Management — lex instructions CLI for syncing AI instructions across IDEs (Copilot, Cursor, etc.)
LexSona Socket — Behavioral memory API (recordCorrection/getRules) exported via @smartergpt/lex/lexsona
Performance — Cached policy module ID lookups for O(1) resolution

LexSona Integration: Lex 2.0.0 provides a public behavioral memory socket (@smartergpt/lex/lexsona) for persona-based workflows. LexSona is a separate private package (v0.2.0+) that consumes this socket to enable offline-capable persona modes. The socket API is stable and documented; LexSona implementation details remain private. Lex itself is persona-agnostic — the socket is a stable integration point for any behavioral engine.

See CHANGELOG v2.0.0 for full release notes.

Quality Metrics

Metric	Value
Test Files	78
Test Suites	23
Source Files	108
Exports	14 subpaths
Schema Version	2

🌟 Why Lex?

Cognitive Architecture for AI Agents

The Challenge: AI coding assistants lose context between sessions. They can't remember what you were working on, why you made certain decisions, or which parts of your codebase are off-limits.

Our Approach: External memory and structured reasoning — the same techniques human experts use to maintain context across complex, long-running projects.

The Components:

Episodic memory — Lex Frames capture what you were doing, blockers, and next actions
Spatial memory — Atlas Frames provide token-efficient architectural context
Policy enforcement — Boundaries and permissions enforced in CI
Orchestration — LexRunner coordinates multi-PR workflows

Why This Matters:

Continuity — Pick up exactly where you left off, every time
Architecture — AI assistants understand your codebase structure
Guardrails — Prevent violations before they happen
Accessibility — Works with any LLM that supports MCP

🤝 Contributing

We welcome contributions! Here's how to get started:

Read the guides:
Pick an issue:
- Good first issues
- Help wanted
Submit a PR:
- Follow commit conventions (imperative mood)
- Include tests and documentation
- GPG-sign your commits

Contributing Guide →

📦 Related Projects

LexRunner — Orchestration for parallel PR workflows
LexSona (v0.2.0+) — Behavioral persona engine, separate private package Consumes Lex behavioral memory socket (@smartergpt/lex/lexsona) to enable offline-capable persona modes with constraint enforcement. High-level concept: persona-driven workflows without embedding persona logic in Lex core. See Control Stack documentation for conceptual framework (public portions).

📄 License

MIT License — Free for personal and commercial use.

See LICENSE for full text.

🔗 Links

Documentation: docs/
Examples: examples/
npm Package: @smartergpt/lex
Issues: GitHub Issues
Discussions: GitHub Discussions

Built with ❤️ by the Lex community

⭐ Star on GitHub · 📦 Install from npm · 💬 Join Discussions

💻 Advanced Topics

TypeScript Build System

Lex uses TypeScript project references for deterministic, incremental builds:

npm run build      # Compile with project references
npm run clean      # Clean build artifacts
npm run typecheck  # Type-check without emitting

Why NodeNext module resolution?

Source uses .ts files with .js import extensions
TypeScript resolves imports during compilation
Emitted .js files work correctly in Node.js ESM
No confusion between source and build artifacts

Build System Details →

Local CI with Docker

Run CI checks locally without touching GitHub:

npm run local-ci          # Run full CI suite locally
npm run local-ci:nonet    # Run without network access

This uses ci.Dockerfile for local parity with CI checks.

Multi-Language Policy Scanning

While TypeScript scanning is built-in, Python and PHP scanners are available as examples:

# Scan Python codebase
python examples/scanners/python/scan.py src/ > python-facts.json

# Scan PHP codebase
php examples/scanners/php/scan.php src/ > php-facts.json

# Merge with TypeScript facts
lex merge ts-facts.json python-facts.json > merged-facts.json

# Check policy
lex check merged-facts.json

⚠️ Security Note: External scanners execute arbitrary code. Review before use.

Scanner Documentation →

Customizing Prompts & Schemas

Lex uses a precedence chain for configuration:

Environment: LEX_CANON_DIR=/custom/canon (highest)
Local overlay: .smartergpt/prompts/
Package defaults: prompts/ (lowest)

# Customize locally
cp prompts/remember.md .smartergpt/prompts/
vim .smartergpt/prompts/remember.md

# Or use custom directory
LEX_CANON_DIR=/my/custom/canon lex remember ...

Environment Variables

Variable	Purpose	Default
`LEX_LOG_LEVEL`	Log verbosity (`silent`, `trace`, `debug`, `info`, `warn`, `error`, `fatal`)	`info` (tests: `silent`)
`LEX_LOG_PRETTY`	Pretty-print logs (`1` = enabled)	Auto-detect TTY
`LEX_POLICY_PATH`	Custom policy file location	`.smartergpt/lex/lexmap.policy.json`
`LEX_DB_PATH`	Database location	`.smartergpt/lex/memory.db`
`LEX_DB_KEY`	Database encryption passphrase (required in production)	None (unencrypted)
`LEX_GIT_MODE`	Git integration (`off`, `live`)	`off`
`LEX_DEFAULT_BRANCH`	Override default branch detection	Auto-detect from git
`LEX_CANON_DIR`	Override canonical resources root	Package defaults
`LEX_PROMPTS_DIR`	Override prompts directory	Package defaults
`LEX_SCHEMAS_DIR`	Override schemas directory	Package defaults
`LEX_CLI_OUTPUT_MODE`	CLI output format (`plain` or `jsonl`)	`plain`
`LEX_BACKUP_RETENTION`	Number of database backups to retain	`7`
`SMARTERGPT_PROFILE`	Profile configuration path	`.smartergpt/profile.yml`

🔐 Database Encryption

Protect your Frame data with SQLCipher encryption:

# Enable encryption for new databases
export LEX_DB_KEY="your-strong-passphrase-here"
lex remember --reference-point "work" --summary "Encrypted!"

# Migrate existing database
lex db encrypt --verify

# Production mode requires encryption
export NODE_ENV="production"
export LEX_DB_KEY="production-passphrase"

Key Features:

✅ AES-256 encryption at rest
✅ PBKDF2 key derivation (64K iterations)
✅ Mandatory in production (NODE_ENV=production)
✅ Migration tool with integrity verification

Security Guide →

Environment Configuration →

🧪 Development

Prerequisites

Node.js: v20+ LTS (tested up to v22, see .nvmrc)
npm: v10+
Git: For branch detection

Local Setup

# Clone repository
git clone https://github.com/Guffawaffle/lex.git
cd lex

# Install dependencies
npm ci

# Build
npm run build

# Run tests
npm test

# Local CI (full suite)
npm run local-ci

Project Structure

lex/
├── src/                     # TypeScript source (no .js files)
│   ├── memory/             # Frame storage & MCP server
│   ├── policy/             # Policy enforcement & scanners
│   ├── shared/             # Shared utilities & types
│   └── index.ts            # Main entry point
├── dist/                    # Build output (gitignored)
├── canon/                   # Canonical prompts & schemas
├── docs/                    # Documentation
├── examples/                # Usage examples & optional scanners
├── test/                    # Test suite
└── .smartergpt/            # Local workspace (gitignored)

Running Tests

npm test                     # Run all tests (excludes git tests)
npm run test:coverage       # With coverage report
npm run test:watch          # Watch mode
npm run test:git            # Git integration tests (requires non-interactive signing)

Note: Git tests are quarantined due to mandatory GPG signing in this environment. See test/README.md for details.

Code Quality

npm run lint                # ESLint checks
npm run format              # Prettier formatting
npm run typecheck           # TypeScript validation