Document Chat

Run it yourself

You need Docker and Ollama on the same computer, with two models pulled (about 6 GB in total). Then:

ollama pull embeddinggemma
ollama pull qwen3:8b
git clone https://github.com/Nbhaskell/rag-project.git
cd rag-project
docker compose up --build

Open http://localhost:8000. The chat there is live and you can ask it anything about the demo documents, or point it at your own folder. It runs entirely on your computer. The README has the details, including a route without Docker.

How it works

1. Documents are cut at their headings (PDFs one page at a time) and turned into lists of numbers by a small search model.
2. A question is turned into numbers the same way and the four closest pieces are found. A question about several things is split, and each part is searched on its own.
3. Only those pieces go to the language model, with strict instructions to answer from them or say "I don't know".
4. The answer comes back with its sources and the exact passages, so you can check it against the original wording.

How well does it work?

Measured on the local model, three repeats of each question, against these made-up documents:

• Every answer was right on 67 development questions, and on 55 held-out questions written after the tuning was finished.
• The search found the text it needed in 59 of 59 checks on the first set and 45 of 46 on the held-out set.
• The weakest part is the Sources line: exactly right about 94% of the time on the first set and 77% to 80% on the held-out set (it often lists one file too many).
• On 9 questions that try to make it break its rules it passed 32 of 45 (five repeats). It resists the obvious ones and follows some hidden orders, as the last recorded question shows.

These tests were written and run by the same person with an AI assistant, and the documents are small and invented, so they show that the method works, not that it scales. The saved results are in the repository.

The documents

All of them are invented, including the names, phone number and address.