Specialism
RAG chatbot development for company documents
An assistant that answers from your own policies, contracts and manuals — showing which document it used, and saying plainly when it does not know. That last behaviour is harder to build than the first, and it is what decides whether anyone trusts the thing.
The method
Retrieval first, generation second
RAG stands for retrieval-augmented generation. Instead of trusting what a model absorbed in training, the system searches your documents for relevant passages and asks the model to answer using only those.
The analogy that lands with most clients: it is the difference between asking a new starter to recall the refund policy from memory and handing them the policy open at the right page. The second is more accurate, stays correct when the policy changes, and lets them point at where the answer came from.
It follows that the quality of one of these systems is mostly the quality of the retrieval, not the sophistication of the model. When people report that an internal assistant "makes things up", the usual cause is not the model inventing facts — it is retrieval handing it the wrong three paragraphs and the model doing its best with them.
Why fine-tuning is usually the wrong answer
Fine-tuning comes up in almost every first conversation, and it is almost always an expensive detour. Fine-tuning teaches a model style and format, not facts. A fine-tuned model still has no idea what changed in a document last week, still cannot cite a source, and has to be retrained whenever the underlying material moves.
Retrieval handles facts, stays current automatically, and produces citations as a by-product. Start there. Fine-tuning is a later optimisation, if it is ever justified at all.
| Component | Effect on answer quality | Most common mistake |
|---|---|---|
| Document preparation | Very high | Indexing everything, including superseded versions |
| Chunking strategy | Very high | Fixed-size splits that cut a clause in half |
| Retrieval and ranking | High | Never tested against a real question set |
| Permission filtering | Critical for safety | Applied after generation instead of at retrieval |
| Refusal behaviour | High for trust | Never tested, so the system answers regardless |
| Model choice | Moderate | Treated as the main lever when it is one of the smallest |
Permissions belong at retrieval. A user's question must only ever search documents that user is entitled to see. Filtering the answer afterwards is not equivalent: by then the restricted content has already shaped the response, which is a disclosure whether or not it is quoted verbatim. Governance is documented against the NIST AI Risk Management Framework.
Delivery
The evaluation set is the real deliverable
Before building, we write down the questions the assistant must answer correctly, with expected answers, gathered from the people who currently field them. Included deliberately are questions it should refuse — things outside its documents, or that it has no business answering.
That set becomes the regression test. Every change to chunking, retrieval, prompting or model gets measured against it rather than judged by whoever last tried the demo. It also survives us: when you change model provider in eighteen months, the evaluation set is what tells you whether the swap helped or quietly degraded things.
Related pages: agent development for assistants that take actions rather than just answer, and document processing for extracting structured data rather than answering questions. The difference between an agent and a chatbot is worth reading before scoping either. When the requirement turns out to be reaching live systems rather than documents — an account balance, an order status, a ticket that needs creating — that is MCP server development, which exposes those systems to an assistant as permissioned tools instead of indexed text.
Questions
What clients ask about retrieval systems
What does RAG actually mean?
Retrieval-augmented generation. Rather than relying on what a model absorbed during training, the system first searches your documents for passages relevant to the question, then asks the model to answer using only those passages. It is the difference between asking someone to recall a policy from memory and handing them the policy open at the right page.
Do we need to fine-tune a model on our data?
Almost never, and it is a common expensive detour. Fine-tuning teaches style and format, not facts, and a fine-tuned model still cannot tell you what changed in a document last week. Retrieval handles facts, stays current as documents change, and lets the system cite its source. Start there; fine-tuning is a later optimisation if ever.
How do you stop it answering when it should not?
By treating that as a first-class requirement rather than an edge case. When retrieval returns nothing sufficiently relevant, the correct behaviour is to say so and offer a handoff. We include unanswerable questions in the evaluation set specifically so refusal is tested, because a system that never says no will confidently invent an answer eventually.
Can it respect who is allowed to see which documents?
Yes, and it has to be designed in from the start. Permissions are applied at retrieval, so a user's query only ever searches documents they are entitled to see. Filtering after generation is not a safe alternative — by then the restricted content has already influenced the answer, which is a leak whether or not it is quoted.
Free consultation
Tell us what is costing you the most time
Describe the process. You get a straight read on whether it is worth automating, roughly what it would take, and what we would tackle first — on a call that costs nothing.
- Under an hour, free, and no obligation
- Fixed price per stage, agreed before any work starts
- You own everything we build, documented and in your accounts
- If you do not need us, we will say so on that call
The form is the fastest way to reach us — we reply from a real person, usually within one business day.