Per-Request Overrides

Most RAG knobs in GENIE.AI are deployment-wide (set in .env). A subset can be overridden on a single chat request via the OpenAI-compatible retriever_parameters and reranker_parameters fields on the request body — useful for an admin tool that wants to A/B a chunk count, for an experiment harness that varies top_n per query, or for an integration test that needs to lock in a known reranker strategy.

This page is for backend developers building integrations against ChatQnA and operators experimenting with retrieval and reranking knobs.

Prerequisites

• Familiarity with Retrieval and Reranking — these override the same knobs.
• The canonical Pydantic models are GenieaiRetrieverParms(RetrieverParms) and GenieaiRerankerParms(RerankerParms) in genie-ai-overlay/chatqna/genieai_chatqna.py:595-602.
• The request must include Authorization: Bearer <keycloak_jwt> and a traceparent header (ChatQnA propagates the trace).

How overrides flow

POST /v1/chatqna
   request.retriever_parameters = {top_k: 2, score_threshold: 0.2}
   request.reranker_parameters  = {top_n: 1, reranking_strategy: "slice"}
        ↓
ChatQnA (chatqna.py:908-941)
   safe_params = model_dump(exclude_unset=True, exclude_none=True)
   inputs.update(safe_params)
        ↓
Retriever microservice / Reranker microservice
   inputs["top_k"] / inputs["top_n"] / ... consumed by that stage

Two important behaviours:

1. exclude_unset=True, exclude_none=True. Any field the caller does not set falls back to the env var / in-code default. You don’t need to repeat the whole schema — send only what you want to override.
2. Encoding the label filter. The label filter is encoded in the standard search_start field ("chunk" / "chunk_label_filter"). Overriding search_start overrides the auto-filter — see encode_filter_labels (defined in genie-ai-overlay/core/label_contract.py, imported and called at genieai_chatqna.py:933-936) and the data contract note in Retrieval → Label filtering.

Retriever overrides

Backed by GenieaiRetrieverParms(RetrieverParms) in genieai_chatqna.py:595. The fields forwarded to the retriever service:

Reranker overrides

Backed by GenieaiRerankerParms(RerankerParms) in genieai_chatqna.py:602. The fields forwarded to the reranker service:

Example: A/B a top-k comparison

# Default env: RETRIEVER_ARANGO_K=4. We want to A/B with top_k=2.
curl -sk -X POST http://localhost:8888/v1/chatqna \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "traceparent: 00-$TRACE_ID-01" \
  -d '{
    "messages": [{"role": "user", "content": "How do I renew my permit?"}],
    "stream": false,
    "retriever_parameters": {
      "k": 2,
      "fetch_k": 12
    },
    "reranker_parameters": {
      "top_n": 1
    }
  }'

The response carries the same source_documents array as the streaming metadata event — compare its length and metadata.score to the un-overridden baseline.

Example: constrain retrieval to a service category

# Filter retrieval to chunks whose labels include "Healthcare".
curl -sk -X POST http://localhost:8888/v1/chatqna \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What are the diabetes management guidelines?"}],
    "stream": false,
    "retriever_parameters": {
      "filter_labels": ["Healthcare"]
    }
  }'

filter_labels is intersected with the labels actually assigned to chunks at ingest time — empty intersection returns no chunks.

Verification

After a per-request override, the result is observable through the source_documents array in the SSE metadata event — ChatQnA does not emit retriever.parameters / reranker.parameters as chatqna span attributes (the overrides flow into the per-request retriever and reranker microservice HTTP calls, but no dedicated span attribute is set on the chatqna.orchestrate root span). The reranker.strategy attribute on the reranker microservice span can confirm the strategy override landed.

Failure modes

Related

• Retrieval — what the retriever does with the overrides.
• Reranking — what the reranker does with the overrides.
• Streaming & metadata events — how the result of the override is surfaced to the backend.
• Backend → API contracts — the BFF route that proxies this.