RAG supplies documentation excerpts to the response and can reduce errors, but does not eliminate them. Responses need evaluation, sources need maintenance, and the system should hand over to a person when information is missing.
Why classic chatbots disappoint
The first generation of web chatbots was a tree of buttons: 'press 1 for hours, 2 for prices'. If your question didn't fit a predefined branch, there was no answer. They were little more than a disguised FAQ, and users hated them.
With the arrival of language models came the opposite problem: chatbots that understand any question, but make up the answer when they don't know it. For a business, a bot that asserts false things about your services or prices is worse than having no bot at all.
Connecting your website directly to a generic model without controls is risky: it can promise discounts that don't exist, invent policies or give outdated information. The user will think they're talking to your company.
What a RAG chatbot is and why it changes everything
RAG stands for Retrieval-Augmented Generation. In plain terms: before answering, the chatbot searches your content for the relevant fragments and only then generates the answer from them.
RAG supplies documentation excerpts to the response and can reduce errors, but does not eliminate them. Responses need evaluation, sources need maintenance, and the system should hand over to a person when information is missing.
RAG supplies documentation excerpts to the response and can reduce errors, but does not eliminate them. Responses need evaluation, sources need maintenance, and the system should hand over to a person when information is missing.
Local RAG: review the complete data flow
There's a second decision as important as RAG itself: where everything lives. In many solutions, your content and your visitors' conversations are sent to third-party servers to be processed. That raises privacy and compliance issues.
A local index does not mean all processing is local. To keep data within your infrastructure, the model, embeddings, logs and other components must also run there. Providers, retention and permissions need review for the specific deployment.
- Define where the model and retrieval run.
- Review providers and the data they receive.
- Configure retention, access and deletion.
- Evaluate accuracy before and after release.
A local index does not mean all processing is local. To keep data within your infrastructure, the model, embeddings, logs and other components must also run there. Providers, retention and permissions need review for the specific deployment.
Local RAG versus generic chatbot: a comparison
Side by side, the difference is clear:
- Source: RAG adds retrieved documentation to the model context.
- Reliability: it can reduce errors but still needs evaluation.
- Privacy: it depends on where each component runs.
- Updates: refresh the index when content changes.
- Cost: includes infrastructure, model, maintenance and evaluation.
What it's actually useful for on a website
A well-built local RAG chatbot stops being an ornament and starts doing useful work:
- Answering product or service questions instantly, at any hour, from your real content.
- Qualifying visitors before passing them to a person, saving sales time.
- Guiding the user to the right page or form.
- Serving each visitor in their own language even if your content is in another.
- Collecting what your visitors ask, a source of content and SEO opportunities.
The hidden value of a RAG chatbot is the conversation log: it tells you exactly what doubts your visitors have before buying. That information is worth as much as the answers it gives.
What you need to implement it
Setting up a local RAG chatbot on a website is simpler than it seems. The essential steps:
- Select what content to index: pages, posts, products, PDFs
- Generate the index (vector store) on your own server or database
- Configure the bot to answer only from that content, without making things up
- Customise name, tone and colours to match your brand
- Enable language detection to answer each visitor in their own
- Enable conversation logging to analyse the questions
- Test with real questions from your sector before publishing it
A local index does not mean all processing is local. To keep data within your infrastructure, the model, embeddings, logs and other components must also run there. Providers, retention and permissions need review for the specific deployment.
We can assess which assistant suits your website and the deployment, content and supervision it needs.