Zeb Wiki
An agent-based knowledge workflow that turns scattered project documents into a structured, searchable wiki, inspired by Andrej Karpathy’s “LLM Wiki” concept.
Explore problem, architecture & my role
01 · Problem
Project knowledge was spread across Teams and SharePoint documents. Finding scope, technologies, project context, or similar past work required manually reviewing multiple files, making knowledge reuse slow.
02 · Solution & architecture
Built an agent-based wiki pipeline inspired by Andrej Karpathy’s LLM Wiki idea: turn source documents into structured entities and persistent Markdown knowledge pages, then make them discoverable through semantic retrieval.
- Entity extraction: identify projects, clients, technology stacks, scope, and other attributes.
- Wiki management: create or update Markdown pages and maintain the schema as new entity types appear.
- Semantic retrieval: store wiki Markdown in AWS S3 and embeddings in PostgreSQL for similar-project discovery.
- Human-in-the-loop: include a human review checkpoint for ambiguous or low-confidence extracted details and proposed knowledge updates before they are accepted.
Reference: Inspired by Andrej Karpathy’s “LLM Wiki” concept, which treats a knowledge base as a maintained, evolving collection of linked notes rather than a one-off document summary.
03 · My role
- Designed and implemented the agentic workflow for transforming source documents into structured wiki knowledge.
- Built the LangChain orchestration flow and connected document processing, entity extraction, similarity detection, and wiki management tools.
- Implemented S3 storage for Markdown pages and PostgreSQL embeddings for semantic search.
- Worked on dynamic entity/schema management and integrated Claude Sonnet 4.5 for document understanding and extraction.
- Evaluated extraction quality and identified agent execution cost as an optimization area.
04 · Results
Created a searchable organizational knowledge base, enabled discovery of similar projects, and reduced time spent locating previous project documentation by 90%. The work also identified cost-efficiency opportunities for production-scale use.
05 · Tech stack
Python · LangChain · AgentCore · Claude Sonnet 4.5 · AWS S3 · PostgreSQL · Embeddings · RAG / vector search · Markdown
