RAG reranking with Jev: how we use it in production

A question comes into the assistant. The agent runs two semantic searches and one keyword search. The result: 19 chunks, taken from 8 different documents. Some answer the question spot on, others are on the right topic without answering it, and a few are only there because a keyword caught them.

Before September 24, 2026, those 19 chunks would have gone straight to the LLM that writes the answer. Since then, Jev, TypeSafe AI's model, scores them one by one first. On that particular search, it kept 8.

This article describes our RAG reranking as it runs today. Where Jev plugs in, then what its first day in production shows about the scores it gives, the chunks it keeps, response time and cost.

1. The problem: too many chunks, not enough signal

A serious enterprise RAG combines several search paths: semantic, keyword, sometimes metadata. That is what lets it find "remote work arrangements" as well as the reference XOD1132 (we covered this in our article on hybrid search).

The downside is mechanical. Each path brings back its own candidates, you merge them, deduplicate them, and end up with more chunks than the LLM can use. The pile contains false positives (the keyword is there, the topic is not), near-duplicates (the same paragraph in three documents) and chunks that circle the topic without answering the question.

The answering LLM can often sort this out on its own. Not always. And when it does, it pays for every token it reads, its attention gets diluted, and it sometimes cites the wrong source. Reranking is the step between retrieval and generation: score each chunk, reorder, keep only the best.

2. Jev, TypeSafe's AI that answers with a probability

Jev was released on September 15, 2026. TypeSafe AI, a San Francisco company, describes it as its first "System One" model: a model that does not generate text. You send it a state (JSON) and closed questions, and it returns typed decisions.

For our use, only one type of question matters: the "noul" question, a yes/no question that Jev answers with the probability that the answer is yes, between 0 and 1.

Here is the question we ask for every chunk, word for word:

Is this chunk relevant to answer, at least partially, at least one of the search queries?

Two details in that sentence matter. "At least partially", because a chunk that provides half of the answer should stay. "At least one of the search queries", because the agent often runs several searches for a single question, and a chunk that helps with one of them should not be dropped because it ignores the others.

On the code side, we locked our Jev integration to this single mode: any attempt to use it for chat, completion, OCR or embeddings raises an error. Jev does one thing on our platform, and only one.

3. Where Jev plugs into our RAG

Jev plays no part in ingestion, indexing or embedding computation. It acts at the very end of document search, on the list of chunks that has already been built, right before that list is handed to the LLM.

Loading diagram…

The diagram follows a document search in the assistant: the agent queries the index by meaning and by keywords, the results are merged and grouped by document, then Jev scores each chunk in parallel. The agent only receives the chunks that were kept, in order of relevance, and writes its answer from them.

For each candidate chunk, we send Jev a state like this:

{
  "semantic_queries": ["notice period when a manager resigns"],
  "keyword_queries": ["notice", "resignation", "manager"],
  "document_name": "Collective agreement 2025.pdf",
  "chunk_content": "Article 12. In the event of resignation, the notice period is set at..."
}

The calls all go out in parallel, one per chunk.

This is where RAG + Jev differs from a classic reranker. Jev receives the semantic queries, the keywords and the document name all at once, and judges the chunk against the whole search.

Once the scores come back, the least relevant chunks are dropped, and documents are reordered by the best score among their chunks. The answering LLM never sees the scores: it simply gets a shorter, better-ordered list.

One last, less visible effect: our "which documents were used for this answer" statistics now only count the chunks Jev kept. A dropped document no longer inflates its usage rate.

4. A score you can read, even for keyword search

Ranking chunks is something search already does. What we want to know is something else: can we answer? Among what came back, is there enough to build a reliable answer?

With semantic search, there is already a hint. The similarity score between the question and the chunk is fairly easy to read: a very close chunk is likely to be relevant, a distant one much less so.

Keyword search is another story. The score depends on the number of occurrences, how rare the terms are and how long the chunk is. It is useful for ranking results of the same query against each other, but its absolute value means little: a score of 12 is not "twice as relevant" as a score of 6, and it cannot be compared from one query to another.

Jev solves this. It asks the same question of every chunk, whichever path found it, and returns a probability between 0 and 1 that reads the same way everywhere. A chunk found by keyword and a chunk found by meaning finally sit on the same scale: how useful they are for answering.

5. Jev in production: scores and chunks kept

Here is what the first day in production, September 24, 2026, shows. A search that went through Jev had between 8 and 20 chunks, 10 on median, drawn from 5 documents on median.

The scores

First observation: scores are rather high. The median score is 0.80, 70% of chunks are above 0.5, and only 2% fall below 0.1. These high scores mostly say that the document search upstream does its job well, since most of what it retrieves is already relevant. All Jev has to do is drop what sticks out.

That does not mean it says "yes" to everything. On more than half of searches, the gap between the best and the worst score is over 0.5, and that gap is what makes sorting possible.

Chunks kept

On average, a search comes in with 10.7 chunks and leaves with 7.7. So Jev drops a little more than one chunk in four, and those are the lowest-scored ones: the ones that would have diluted the answer the most.

6. Response time: what Jev adds

All the calls of a search go out at the same time. The full search therefore takes barely longer than the slowest call, and the added time does not depend on the number of chunks: a 20-chunk search waits no longer than an 8-chunk one.

It does depend on chunk length: the longer a chunk, the longer Jev takes to read it. Reranking therefore adds a short delay to document search. We think we can still bring it down, and that work is under way.

And if Jev does not answer, the assistant still does: it receives the full list of chunks, just as it did before September 24.

7. What it costs

Jev is billed on input tokens: $0.042 per million tokens. The output, a probability, is negligible.

On the first day, with chunks averaging 1,600 tokens, that comes to $0.0007 per search, or about $0.70 per 1,000 searches.

At that price, the bill is irrelevant. What we keep an eye on is the added time.

Every call is still logged in our monitoring (provider, task, tokens, cost, organization, conversation), like any other LLM call.

8. Why Jev rather than another reranker

Jev is not our first attempt at reranking. The solutions we had tested before did not offer this balance between cost, latency and quality.

Beyond the numbers, the difference for us comes down to the question. A classic reranker applies the relevance it has learned. With Jev, we write it ourselves: "at least partially", "at least one of the search queries". If we want to drop outdated chunks tomorrow, we will only need to pass the document date and adjust the sentence, with no model to retrain.

9. Why Jev is only active under the Worldwide policy

Each client organization chooses its AI policy: EU-only or Worldwide.

TypeSafe AI is an American company. Sending document chunks to Jev means transferring client content to a provider outside the European Union. So we restricted it to the Worldwide policy, like our other non-European providers.

In practice, in our routing table, the "chunk relevance" task only exists for the Worldwide policy. An EU-only organization has no Jev reranking: its search works exactly as it did before September 24. There is no box to tick, so no risk of forgetting it: for these organizations, the task does not exist.

It is the same reasoning we apply everywhere, and that we detailed in our article on what sits behind the word "sovereign": where a server sits is not enough, you have to look at who operates the model.

10. How we validated it before switching it on

We did not put Jev into production on the strength of an announcement. Real-world use of Jev was first simulated outside production:

1. a script reads real past document searches from our database, with exactly the list of chunks the LLM had received;
2. it asks Jev the same question again, chunk by chunk;
3. it produces a report (spreadsheet and PDF) that lines up each chunk, its score and the keep / drop decision;
4. these reports were reviewed by hand, chunk by chunk, to check that what was dropped deserved to be.

These simulations do not feed production monitoring, so they skew neither our costs nor our statistics. Our selection rules come from that review.

And going live remains reversible: a switch turns reranking off for the whole platform, without touching the code.

The Ask This Guy approach

Jev shows how we integrate a new model: a single role, a precise place in the chain, a fallback if it does not answer, call-by-call monitoring and respect for each client's AI policy.

Our clients have nothing to configure. If their organization is on the Worldwide policy, their document searches have been filtered by Jev since September 24. If it is EU-only, nothing changes for them. The documentation details which data goes to which providers.

Frequently asked questions about RAG reranking with Jev

What is reranking in RAG?

Reranking is a second sorting step between retrieval and generation. Retrieval (semantic, keyword or both) brings back a broad list of candidates. The reranker scores each of them against the question, reorders them and, in our case, drops the least relevant ones. The answering LLM then works on a shorter, better-ordered context.

Is Jev an LLM?

No. Jev does not generate text. It receives a structured state and closed questions, and returns typed answers, for example a probability for a yes/no question. That is what makes it a good fit for reranking: it is not asked to write, only to decide whether a chunk is relevant.

Does Jev replace vector search?

No. Jev has no access to the index and does not search for anything. It scores chunks that semantic and keyword search have already found. Without good retrieval upstream, it has nothing good to keep.

How many chunks does Jev keep?

On our first day in production, a search came in with 10.7 chunks on average and left with 7.7. Jev therefore drops a little more than one chunk in four, starting with the lowest-scored ones.

How much does Jev cost for reranking?

Jev is billed $0.042 per million input tokens. On our first day in production, with chunks averaging 1,600 tokens and a median of 10 chunks per search, that comes to about $0.0007 per search, or $0.70 per 1,000 searches.

How much response time does Jev add to a RAG?

A short delay. All the calls of a search go out in parallel, so the added time depends on chunk length, not on the number of chunks. If Jev does not answer, the assistant still answers, using the full list of chunks.

Can Jev be used with a European data requirement?

TypeSafe AI is an American company. At Ask This Guy, Jev is therefore only active for organizations on the Worldwide AI policy. Organizations on the EU-only policy send no chunks to Jev, and their document search works without this reranking step.

---

Conclusion

One day in production is not a final verdict, but it already tells us the essentials. Jev does what we ask of it: it drops more than one chunk in four and gives every chunk a readable score, at negligible cost. Its limit is the time it adds to the answer. That is where our next work will go, and we will publish the numbers once it is in place.

If you want to see what a filtered document search looks like on your own documents, book a demo. The full approach is on the turnkey enterprise RAG page.