All work
RAG
Local RAG Agent
Cited Q&A over your own files that runs entirely on your machine: no API keys, nothing sent anywhere.
The problem
Many organizations can't send their documents to a cloud model, yet still need answers from long regulations and policies, with the exact article cited.
How it works
Documents are parsed with their pages and tables, chunked by structure (articles, chapters, annexes) and indexed in ChromaDB. Questions run vector search and BM25 in parallel, fused with RRF and optionally reranked, then a local model on Ollama answers with file and page citations. Re-indexing is incremental, and a built-in evaluation reports Hit@k and MRR.
Pipeline
- 01Load
- 02Clean
- 03Chunk by structure
- 04Hybrid search
- 05Local LLM + citations
Where it's used
- Built as a Spanish assistant for the rules of Peru's Beca 18 national scholarship
- Public institutions and NGOs answering questions about regulations
- Any team whose documents must stay on-premises
Highlights
- Fully offline with Ollama, no data leaves the machine
- Legal-aware chunking that keeps articles whole, with breadcrumbs
- Fixed refusal when nothing relevant is retrieved
- Incremental re-indexing and built-in retrieval evaluation (Hit@k, MRR)
Stack
OllamaChromaDBBM25Cross-encoderPython
Want something like this for your company?
I build production versions of these systems on your data and your infrastructure.
Related service: Knowledge assistants
Contact me