The problem

Most organisations hold their working knowledge in places people cannot easily search: policy PDFs, HR handbooks, procedure manuals, contract repositories, ticket histories, and internal wikis. The information exists, but finding the right passage and trusting it takes time. Staff ask colleagues, guess, or act on a document that has since been superseded.

A general chatbot makes this worse in a specific way. It answers fluently whether or not it has grounds to, invents citations, and has no concept of who is permitted to read a given document. For internal knowledge this is not acceptable: an answer without a verifiable source is a liability, and an answer that quotes a document the user was never cleared to see is a breach.

The problem is sharper in bilingual Arabic and English settings. Queries and source documents mix scripts and directions, Arabic morphology defeats naive keyword search, and many off-the-shelf embedding models handle Arabic poorly. An assistant that works well in English can be close to useless on the same organisation’s Arabic material.

What I design and build

I design retrieval-augmented assistants that answer from an organisation’s own documents and databases, with every answer grounded in retrieved passages and shown alongside its citations. The user sees not only the answer but the specific source it came from, so they can verify it themselves.

The core is a hybrid retrieval pipeline: lexical search for exact terms, names, and clause numbers, combined with dense semantic search for meaning, followed by a reranking stage that reorders candidates before anything reaches the model. Retrieved passages are the only material the assistant is instructed to answer from. When retrieval returns nothing relevant, the assistant is built to say so rather than fill the gap.

For Arabic and mixed-language content I select embedding and retrieval components suited to the language rather than assuming English defaults carry over — multilingual embedding models such as BGE-M3 and multilingual-e5, with Arabic-aware processing through CAMeL Tools where normalisation and tokenisation matter. Access control is part of the retrieval layer, not an afterthought bolted on top.

Where it fits

This approach fits internal knowledge that is large, changes over time, and needs to be answered with a source: policy and compliance libraries, HR and operations manuals, product and support documentation, and structured records queried in natural language.

It is not the right tool for every problem. If the underlying documents are contradictory, out of date, or badly maintained, an assistant will surface that mess faithfully rather than fix it; the honest first step there is often document cleanup, not AI. If a task needs exact computation, deterministic business logic, or a guaranteed answer every time, a retrieval assistant is the wrong instrument and I will say so. It also adds little where a simple keyword search over a small, stable document set already works.

How it works

A query is processed and sent through both lexical and dense retrieval against the indexed documents. The combined candidate set is reranked so the most relevant passages rise to the top. A permission filter removes anything the requesting user is not entitled to see, applied before generation so restricted content never reaches the model. The model then composes an answer constrained to the retrieved passages and returns it with citations pointing back to the source. If retrieval surfaces nothing relevant, the assistant reports that rather than answering from general knowledge.

Model routing sits underneath this: I match the task to the model, using a frontier API where reasoning quality justifies it and an open-weight model where cost, latency, or on-premises requirements point that way.

Security and privacy

Access control is enforced at retrieval time so users only ever receive answers grounded in documents they are cleared to read. Where data cannot leave the organisation’s boundary, the system can run fully on-premises or air-gapped using open-weight models, with no external API calls. Secrets and credentials are held outside the codebase and injected at runtime. Every query and the sources used to answer it can be logged for audit, so there is a record of what was asked, what was retrieved, and what was returned. I treat that audit trail as a first-class requirement, not an add-on.

Evaluation and reliability

I do not ship an assistant on the strength of a few good demo questions. I build a golden set of representative queries with known correct answers and expected sources, then measure the system against it. Retrieval quality is measured directly — for example, recall@k measured before and after adding reranking, so the contribution of each component is visible rather than assumed. Answer quality is assessed with a combination of RAGAS metrics, LLM-as-judge scoring, and human review on a sampled set, checking that answers are faithful to their sources and that citations actually support the claims.

Failure handling is explicit. The assistant is built to abstain when retrieval is weak, to surface low-confidence answers as such, and to avoid fabricating citations. Regression tests run against the golden set so that a prompt change, a model swap, or a document update does not silently degrade quality.

Deployment

The system is packaged in containers and can run in the organisation’s cloud, in a private VPC, or entirely on-premises. It integrates with existing systems of record — document stores, databases, identity providers — rather than requiring content to be migrated into a new silo. Indexing runs on a schedule or on document change so the assistant stays current. Where an identity provider governs access, the assistant’s permission layer is wired to it so that entitlements stay in one place.

How an engagement starts

I start with a scoped assessment rather than a large build. Over a short, fixed engagement I review a representative slice of your documents, define a golden set of real questions, stand up a working retrieval prototype against that slice, and measure it. You get a concrete read on retrieval quality, a realistic view of what your content will and will not support, and a plan — with the option to stop there if the numbers do not justify going further.

For a concrete example of retrieval over demanding bilingual legal text, see the Saudi Labor Law RAG case study, which covers Arabic-aware retrieval, reranking, and evaluation on legal source material.

FAQ

How do you stop it from making things up?

The assistant is constrained to answer only from retrieved passages and to cite them, and it is built to decline when retrieval finds nothing relevant. I measure faithfulness against a golden set with RAGAS, LLM-as-judge, and human review, and I test for the specific failure of citations that do not support their claim.

Can this run without sending our documents to an external API?

Yes. Using open-weight embedding and generation models, the system can run fully on-premises or air-gapped with no external calls. That choice affects model selection and hardware, which I scope during the assessment.

Does it actually work in Arabic, or just English?

I select embedding, tokenisation, and retrieval components for the language rather than assuming English defaults transfer. I evaluate Arabic and mixed-language queries on your own material as part of the golden set, so performance in Arabic is measured, not assumed.

How do you keep users from seeing documents they should not?

Permissions are enforced at retrieval time, before anything reaches the model, and can be wired to your existing identity provider. Restricted content is filtered out of the candidate set, so it never contributes to an answer.