EDITH: Building Semantic Search Over Personal Knowledge

The Problem

I have 200+ markdown notes across two Obsidian vaults:

• Forge: Work notes — meeting summaries, design docs, troubleshooting logs, 187 files
• Cerebro: Personal notes — learning, projects, references, 64 files

Obsidian is great for writing and linking notes. The graph view is beautiful. But finding information is painful.

The built-in search:

• Exact string match (no semantic understanding)
• Case-sensitive by default
• Can’t search across tags/metadata easily
• No ranking (results are chronological)

What I actually need:

“Show me notes about setting up Tailscale”

• Should match “configuring tailscale mesh network”
• Should match “VPN setup using Tailscale”
• Should match files tagged #networking that mention “mesh”

This is a semantic search problem. I need:

1. Embed each note as a vector (captures meaning, not just keywords)
2. Embed my query as a vector
3. Find the nearest neighbors (cosine similarity)
4. Return relevant chunks with context

Existing tools (Obsidian plugins, Notion AI, Roam Research) send your notes to external APIs. That’s a non-starter for work notes — I have Amazon-internal docs, service names, architecture diagrams. Can’t send that to OpenAI.

Solution: Build a local RAG system.

What EDITH Is

EDITH: Enhanced Digital Intelligence Through Heuristics (yes, I backronymed it)

It’s a RAG (Retrieval-Augmented Generation) pipeline that:

1. Chunks markdown files by heading structure (~500 tokens per chunk)
2. Generates embeddings locally using nomic-embed-text (768-dim vectors)
3. Builds a FAISS index (fast approximate nearest neighbor search)
4. Exposes semantic search via an MCP (Model Context Protocol) server
5. Lets Claude (or any LLM) query my notes conversationally

Key constraints:

• Local embeddings: No external API calls (privacy for work notes)
• Fast: Sub-200ms search latency (FAISS vector search)
• MCP-native: Exposes tools Claude can call directly
• Cross-platform: Works on Mac (launchd) and Linux (systemd)

Architecture

1. Markdown Chunking

Obsidian notes are markdown files. Some are 10 lines, some are 1,000. A note is too big to embed as a single vector (you lose granularity), but too small to split by paragraph (you lose context).

Solution: Chunk by heading hierarchy.

Given a markdown file:

# Design: Order Processing Service

## Context
We need to decouple order creation from payment processing.

## Proposal
Use an event-driven architecture with SQS queues.

### Queue Design
- Order created → `orders-created` queue
- Payment processed → `payments-completed` queue

## Alternatives Considered
We could use Kafka, but SQS is simpler.

Chunking produces:

Chunk ID	Heading Path	Content	Token Count
chunk_1	Design: Order Processing Service > Context	”We need to decouple order creation…“	87
chunk_2	Design: Order Processing Service > Proposal	”Use an event-driven architecture…“	112
chunk_3	Design: Order Processing Service > Proposal > Queue Design	”- Order created → orders-created…“	143
chunk_4	Design: Order Processing Service > Alternatives Considered	”We could use Kafka…“	68

Why this works:

• Headings provide natural semantic boundaries (each section is a coherent idea)
• The heading path gives context (you know this chunk is under “Proposal > Queue Design”)
• Chunks are sized for embedding models (100-600 tokens, avg ~350)

2. Local Embeddings: nomic-embed-text

I need an embedding model that:

• Runs locally (no API calls)
• Produces high-quality vectors (captures semantic meaning)
• Works with long context (up to 8K tokens per chunk)
• Fast inference (< 100ms per chunk)

Choice: nomic-embed-text

Specs:

• 768-dimensional vectors
• Trained on 235M text pairs
• Optimized for retrieval tasks
• Runs via Ollama (local inference server)

Why not OpenAI text-embedding-3-small?

• Costs $0.02 per 1M tokens (200+ notes × 5 chunks/note × re-indexing = adds up)
• Requires internet + API key
• Sends my work notes to OpenAI (unacceptable)

Why not Sentence-BERT (all-MiniLM-L6-v2)?

• 384-dim vectors (half the dimensionality = worse retrieval quality)
• Trained on smaller dataset
• Slower inference on CPU

nomic-embed-text via Ollama gives me:

• Free (runs on my Mac Mini M4 Pro)
• Private (never leaves my machine)
• Fast (M4 Pro GPU acceleration, ~40ms per chunk)

3. FAISS Index

Once I have 1,435 chunks (Forge vault) + 280 chunks (Cerebro vault) = 1,715 total chunks embedded as 768-dim vectors, I need fast search.

Brute-force cosine similarity would work but is slow:

• Query: 768-dim vector
• Compare to 1,715 stored vectors (cosine similarity for each)
• Sort by similarity, take top-k

On CPU, this takes ~15ms for 1,715 vectors. Acceptable, but doesn’t scale. If I have 10K notes, brute-force becomes 100ms+.

FAISS (Facebook AI Similarity Search) is faster:

• Builds an index (one-time cost)
• Uses approximate nearest neighbor (ANN) search (trades accuracy for speed)
• For 1,715 vectors, search takes ~2ms

Index type: IndexFlatIP (Flat index with Inner Product)

Why not IndexIVFFlat or IndexHNSW?

• IVF requires training (overkill for 1,715 vectors)
• HNSW has better recall but higher memory usage

Flat index is fine up to ~100K vectors. Beyond that, switch to HNSW.

Persistence:

FAISS index is saved to disk:

~/.edith/
├── forge.index       # FAISS index for work vault
├── cerebro.index     # FAISS index for personal vault
├── forge.metadata    # Chunk metadata (file path, heading, token count)
└── cerebro.metadata  # Chunk metadata

On startup, EDITH loads the index into memory (~10MB for 1,715 vectors). Queries are RAM-only (no disk I/O).

4. MCP Server

EDITH exposes semantic search as an MCP (Model Context Protocol) server. Claude (or any MCP-compatible LLM) calls EDITH like a function.

MCP Tools:

{
  "tools": [
    {
      "name": "search_knowledge",
      "description": "Semantic search across Obsidian vaults",
      "parameters": {
        "query": "string (user's question)",
        "vault": "string (optional: 'forge', 'cerebro', or 'all')",
        "top_k": "int (default: 5, max: 20)"
      }
    }
  ]
}

Example interaction:

User: “How did I set up Tailscale?”

Claude calls:

{
  "tool": "search_knowledge",
  "parameters": {
    "query": "Tailscale setup",
    "vault": "all",
    "top_k": 5
  }
}

EDITH returns:

{
  "results": [
    {
      "vault": "cerebro",
      "file": "home-lab/networking.md",
      "heading": "VPN Setup > Tailscale Configuration",
      "content": "Installed tailscale on Mac Mini, ran `tailscale up --advertise-routes=10.0.0.0/24`...",
      "similarity": 0.87
    },
    {
      "vault": "forge",
      "file": "onboarding/network-access.md",
      "heading": "VPN Options > Tailscale vs OpenVPN",
      "content": "Tailscale is simpler than OpenVPN for mesh networks...",
      "similarity": 0.82
    }
  ]
}

Claude synthesizes: “You set up Tailscale on your Mac Mini by running tailscale up --advertise-routes=10.0.0.0/24. You documented this in home-lab/networking.md. You also compared Tailscale to OpenVPN in your work notes.”

Why MCP?

MCP is an open protocol for LLMs to call external tools. Benefits:

• LLM-agnostic (works with Claude, GPT-4, Llama, etc.)
• Client-side execution (EDITH runs on my machine, not a cloud service)
• Standardized (no custom API to maintain)

Alternatives:

• LangChain: Too heavyweight, Python-specific, abstracts too much
• LlamaIndex: Same issues, tightly coupled to OpenAI
• Custom REST API: Reinventing the wheel, no client libraries

MCP gives me a standard interface with existing client support.

5. Watcher Daemon

Notes change frequently. I add new files, edit existing ones, delete drafts. The FAISS index needs to stay in sync.

Solution: Filesystem watcher.

On Mac: fswatch monitors vault directories. On Linux: inotifywatch.

# Launch daemon (runs in background)
fswatch -o ~/Dropbox/cerebro ~/Forge/wiki | while read; do
  edith-reindex
done

When a file changes:

1. Watcher detects modification
2. Re-chunk the file
3. Re-embed chunks
4. Update FAISS index (incremental)
5. Save to disk

Incremental indexing:

Instead of rebuilding the entire index, EDITH:

1. Computes diff (which chunks changed)
2. Removes old chunks from index
3. Adds new chunks to index
4. Saves updated index

For 1-2 file changes, this takes ~500ms (vs. 30 seconds to rebuild from scratch).

6. Cross-Platform Service Management

EDITH runs as a background service:

macOS (launchd):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "...">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.kpatel.edith</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/local/bin/edith</string>
    <string>serve</string>
  </array>
  <key>RunAtLoad</key>
  <true/>
  <key>KeepAlive</key>
  <true/>
</dict>
</plist>

Linux (systemd):

[Unit]
Description=EDITH MCP Server
After=network.target

[Service]
Type=simple
User=patkarma
ExecStart=/usr/local/bin/edith serve
Restart=always
RestartSec=10

[Install]
WantedBy=default.target

Start/stop:

# macOS
launchctl load ~/Library/LaunchAgents/com.kpatel.edith.plist
launchctl unload ~/Library/LaunchAgents/com.kpatel.edith.plist

# Linux
systemctl --user start edith
systemctl --user stop edith

Implementation Details

Chunking Logic

import re

def chunk_markdown(content: str, filepath: str) -> list[Chunk]:
    """Split markdown by headings, preserve hierarchy."""
    lines = content.split('\n')
    chunks = []
    current_chunk = []
    current_heading = []
    
    for line in lines:
        if re.match(r'^#{1,6}\s', line):  # Heading line
            if current_chunk:
                chunks.append(Chunk(
                    filepath=filepath,
                    heading_path=' > '.join(current_heading),
                    content='\n'.join(current_chunk),
                    tokens=estimate_tokens('\n'.join(current_chunk))
                ))
                current_chunk = []
            
            # Update heading stack
            level = len(re.match(r'^(#+)', line).group(1))
            heading = line.lstrip('#').strip()
            current_heading = current_heading[:level-1] + [heading]
        else:
            current_chunk.append(line)
    
    # Flush last chunk
    if current_chunk:
        chunks.append(Chunk(
            filepath=filepath,
            heading_path=' > '.join(current_heading),
            content='\n'.join(current_chunk),
            tokens=estimate_tokens('\n'.join(current_chunk))
        ))
    
    return chunks

Embedding Generation

import ollama

def embed_text(text: str) -> list[float]:
    """Generate 768-dim embedding using nomic-embed-text."""
    response = ollama.embeddings(
        model='nomic-embed-text',
        prompt=text
    )
    return response['embedding']  # 768-dim list

FAISS Indexing

import faiss
import numpy as np

def build_index(embeddings: list[list[float]]) -> faiss.Index:
    """Build FAISS index from embeddings."""
    dim = len(embeddings[0])  # 768
    vectors = np.array(embeddings).astype('float32')
    
    # Normalize for cosine similarity (inner product on unit vectors = cosine)
    faiss.normalize_L2(vectors)
    
    # Build flat index
    index = faiss.IndexFlatIP(dim)
    index.add(vectors)
    
    return index

def search_index(index: faiss.Index, query_vector: list[float], top_k: int = 5):
    """Search index for nearest neighbors."""
    query = np.array([query_vector]).astype('float32')
    faiss.normalize_L2(query)
    
    similarities, indices = index.search(query, top_k)
    return indices[0], similarities[0]  # Top-k chunk IDs and scores

MCP Server

from mcp.server import MCPServer, Tool

app = MCPServer()

@app.tool()
def search_knowledge(query: str, vault: str = 'all', top_k: int = 5):
    """Semantic search across Obsidian vaults."""
    # Embed query
    query_vector = embed_text(query)
    
    # Search index
    indices, similarities = search_index(
        index=get_index(vault),
        query_vector=query_vector,
        top_k=top_k
    )
    
    # Fetch chunk metadata
    results = []
    for idx, sim in zip(indices, similarities):
        chunk = metadata[idx]
        results.append({
            'vault': chunk.vault,
            'file': chunk.filepath,
            'heading': chunk.heading_path,
            'content': chunk.content,
            'similarity': float(sim)
        })
    
    return results

if __name__ == '__main__':
    app.run(port=3000)

Stats

After indexing both vaults:

Metric	Forge (Work)	Cerebro (Personal)	Total
Files	187	64	251
Total tokens	342,000	89,000	431,000
Chunks	1,435	280	1,715
Avg tokens/chunk	238	318	251
Index size (disk)	8.2 MB	1.9 MB	10.1 MB
Indexing time	23 sec	6 sec	29 sec
Query latency	1.8 ms	0.4 ms	2.2 ms

Observations:

• Work notes (Forge) are denser — more short snippets, meeting logs, quick references
• Personal notes (Cerebro) have longer chunks — essays, deep dives, project logs
• FAISS search is fast (2ms for 1,715 vectors)
• Indexing is slow (29 seconds for 251 files) but only happens once per file change

Example Queries

Query 1: “How do I set up Tailscale?”

Results:

1. cerebro/home-lab/networking.md (similarity: 0.91)

   • Heading: Networking > Tailscale Setup
   • Content: “Installed Tailscale via Homebrew: brew install tailscale. Ran tailscale up --advertise-routes=10.0.0.0/24 to share local network. Added subnet routes in admin console.”

2. forge/onboarding/vpn-setup.md (similarity: 0.84)

   • Heading: VPN Options > Tailscale
   • Content: “Tailscale is the preferred VPN for remote access. Download from tailscale.com, authenticate with your corp account. No manual config needed — mesh network auto-configures.”

Claude’s response:

“You set up Tailscale by installing it via Homebrew (brew install tailscale), then running tailscale up --advertise-routes=10.0.0.0/24 to share your local network. You also documented this for work onboarding — Tailscale is the preferred VPN, and it auto-configures as a mesh network.”

Query 2: “What were the alternatives I considered for the order processing service?”

Results:

1. forge/designs/order-processing.md (similarity: 0.89)
   • Heading: Design: Order Processing Service > Alternatives Considered
   • Content: “We could use Kafka instead of SQS. Kafka has better throughput and exactly-once semantics. But SQS is simpler to operate, and we don’t need Kafka’s ordering guarantees across partitions.”

Claude’s response:

“You considered Kafka for the order processing service but chose SQS because it’s simpler to operate. You didn’t need Kafka’s cross-partition ordering guarantees.”

Query 3: “Show me notes about EV charging”

Results:

1. cerebro/vehicles/ev-charging-guide.md (similarity: 0.93)

   • Heading: EV Charging > Level 2 Installation
   • Content: “Installed Emporia Classic 48A charger. Required 60A breaker, #6 AWG wire, dedicated 240V circuit. Total cost $1,350 including electrician labor.”

2. cerebro/home/power-backup.md (similarity: 0.78)

   • Heading: Home Power > Backup for EV Charger
   • Content: “EV charger is on a separate circuit from backup power. It draws 11 kW — too much for the UPS system. In a power outage, charge the car at a public station.”

Claude’s response:

“Your EV charging setup uses an Emporia Classic 48A charger on a 60A breaker with #6 AWG wire. It cost $1,350 installed. The charger isn’t on backup power — it draws 11 kW, which exceeds your UPS capacity. During outages, you’d charge at public stations.”

Why Local Matters

I could use Notion AI, Mem.ai, or Obsidian’s Copilot plugin. They all offer semantic search. But they send your notes to external APIs.

Privacy concerns:

• Work notes contain Amazon-internal service names, architecture diagrams, oncall runbooks
• Personal notes contain financial data, medical records, private journal entries
• I don’t want any of this leaving my machine

Compliance concerns:

Amazon policy: “Don’t upload internal docs to external AI services without approval.”

Getting approval takes 2 weeks, requires justification, and usually gets denied. Building EDITH locally bypasses this entirely.

Cost concerns:

OpenAI embeddings: $0.02 per 1M tokens. For 431K tokens × re-indexing 5x/week = $0.04/week = $2/year.

Sounds cheap, but:

• Adds up over time
• Requires internet connection
• Rate-limited (annoying during bulk indexing)

Local embeddings via Ollama: free, unlimited, no rate limits.

Lessons Learned

1. Chunk size matters

I started with 1,000-token chunks. Search quality was poor — chunks were too broad, query results weren’t specific enough.

Reduced to 500 tokens → much better. Chunks are focused on a single idea.

Tried 200 tokens → too narrow. Lost context (e.g., a code snippet without the explanation above it).

Sweet spot: 300-600 tokens per chunk.

2. Normalize vectors for cosine similarity

FAISS IndexFlatIP computes inner product, not cosine similarity. If vectors aren’t normalized, results are wrong.

Fix: Always call faiss.normalize_L2(vectors) before indexing or querying.

3. Metadata storage is critical

Early version stored only embeddings in FAISS. When search returned chunk IDs, I had to look up the file path, heading, content separately. This was slow.

Fix: Store metadata in a separate file (metadata.json) with the same index order as the FAISS index. Now chunk ID maps directly to metadata.

4. Incremental indexing is mandatory

Re-indexing 251 files takes 29 seconds. That’s fine once, but not every time I save a file.

Fix: Track which files changed (via watcher), only re-index those files, update FAISS index incrementally.

Now updates take ~500ms instead of 29 seconds.

5. MCP is immature but promising

MCP is a new protocol (released mid-2025). Tooling is rough:

• Python MCP library has bugs (had to patch server.py twice)
• Error messages are cryptic
• No official debugger (had to build my own logging middleware)

But the concept is right. Once tooling matures, MCP will be the standard for LLM tools.

What’s Next

EDITH works well for ~250 files. Future improvements:

1. Expand to 1,000+ files

As I add more notes, I’ll need:

• HNSW index (faster search for large datasets)
• Chunking refinements (handle code blocks better, preserve bullet lists)
• Deduplication (if two notes have identical sections, don’t index twice)

2. Hybrid search

Pure semantic search misses exact matches. If I search “Tailscale IP address”, I want notes that contain IP addresses, not just “Tailscale setup.”

Fix: Combine vector search (semantic) with keyword search (BM25). Return union of both result sets, re-rank.

3. Contextual chunks

Right now, each chunk is independent. But notes have structure — a chunk under “Alternatives Considered” is related to chunks under “Proposal.”

Fix: Encode chunk graph (which chunks are siblings, children, parents). Use graph structure to expand search results.

4. Auto-tagging

I manually tag notes with #networking, #homelab, etc. But I forget, so coverage is spotty.

Fix: Auto-suggest tags based on content. Run an LLM over each note, generate 3-5 tags, let me approve/reject.

5. Web UI

MCP is great for LLM access. But sometimes I want to search manually.

Fix: Build a lightweight web UI (FastAPI + React) that wraps the EDITH MCP server. Bonus: visualize similarity scores, show chunk context.

Conclusion

EDITH is a local RAG system for personal knowledge. It indexes Obsidian vaults, generates embeddings locally, builds a FAISS index, and exposes semantic search via MCP.

Key decisions:

• Chunk by heading hierarchy → natural semantic boundaries
• Local embeddings (nomic-embed-text via Ollama) → privacy, free, fast
• FAISS for vector search → 2ms latency for 1,715 vectors
• MCP server architecture → LLM-agnostic, client-side execution
• Filesystem watcher for incremental indexing → stay in sync with vault changes

Results:

• 251 files, 1,715 chunks, 431K tokens indexed in 29 seconds
• Query latency: 2ms (vector search) + 10ms (LLM call) = 12ms total
• Privacy-preserving: nothing leaves my machine
• Works with Claude, GPT-4, Llama, or any MCP-compatible LLM

Semantic search over personal knowledge isn’t a nice-to-have — it’s infrastructure. Obsidian gives me a place to write. EDITH gives me a way to find what I wrote.

Now when Claude asks, “How did you set up X?”, it queries EDITH and cites my own notes.

That’s the goal.