Deployment Topologies

This page is the single source of truth for which deployment topology fits which use case. It maps every supported combination of Docker Compose profiles, Swarm node labels, GPU presence, observability, and TLS to a named topology with a command, a diagram, and a verification recipe.

For the procedure to deploy a specific topology, follow the linked how-to guide. This page answers “which topology should I pick?” and “what runs where?”.

Goal

Pick the right topology for your hardware and requirements in under five minutes, then jump to the linked procedure to deploy it.

Prerequisites

• Read the Architecture Overview — the C4 container diagram and service auth matrix are referenced throughout.
• Decide on hardware: a single host, a Swarm cluster, or a separate GPU node.
• Decide on AI scope: pure core (no RAG), local OPEA + GPU, or remote GPU node.
• Decide on observability: enabled or disabled (default disabled).
• Decide on TLS: self-signed (default), manual, or Let’s Encrypt.

Topology decision tree

flowchart TD
    Q1{Have GPU on the app host?}
    Q1 -- "No, CPU only" --> CPU[CPU-only topology<br/>DEPLOY_OPEA=0]
    Q1 -- "Yes, single host" --> SNG[Single-host topology<br/>docker compose up + profiles opea,gpu-models]
    Q1 -- "Yes, separate host" --> REM[Remote GPU topology<br/>GPU_NODE_HOST set, GPU_MODEL_REPLICAS=0]
    Q1 -- "Yes, Swarm cluster" --> Q2{One GPU node or shared?}
    Q2 -- "Dedicated GPU node" --> SWARM3[3-node Swarm topology]
    Q2 -- "GPU + app on same node" --> SWARM2[2-node Swarm topology]

    CPU --> OBS{Enable observability?}
    SNG --> OBS
    REM --> OBS
    SWARM3 --> OBS
    SWARM2 --> OBS

    OBS -- "Yes" --> OBSV[Add --profile observability / ENABLE_OBSERVABILITY=1]
    OBS -- "No" --> COR[Default core]
    OBSV --> TLS{TLS source?}
    COR --> TLS
    TLS -- "Self-signed" --> SS[Default — auto-generated]
    TLS -- "Let's Encrypt" --> LE[--profile letsencrypt / CERTBOT_REPLICAS=1]
    TLS -- "Manual certs" --> MAN[Copy to secrets/ssl/]

Node-label matrix (Swarm)

GENIE.AI places every service on a Swarm node by label. The labels gateway, genieai, gpu are applied with docker node update --label-add and consumed by placement.constraints in docker-compose.yaml.

A single node can carry multiple labels — most production topologies run two or three labels on the same physical host.

The OTel Collector (otel-collector) is a special case: it runs in mode: global with no placement constraint, so one collector runs on every Swarm node. This is required so the fluentd logging driver (localhost:24224 on every node) has a collector to forward logs to.

Note on chatqna-xeon-ui-server / chatqna-xeon-nginx-server: both have profiles: [opea] and replicas: 0 by default — they are disabled in both docker compose up and docker stack deploy. They are listed in the gpu=true column above for completeness but do not run unless you explicitly raise replicas: 1 (Swarm) or use --profile opea (Compose dev). They are debug helpers, not part of the active RAG pipeline.

Profile matrix (Compose)

Docker Compose profiles control which optional services start. They are honored by docker compose up and ignored by docker stack deploy (Swarm starts everything regardless of profile — gating is done via env vars like DEPLOY_OPEA / ENABLE_OBSERVABILITY / CERTBOT_REPLICAS).

Profile vs Swarm gotcha: when you run docker stack deploy, Swarm reads image: (never build:), ignores profiles:, and uses replicas: / env-var expansion to decide what runs. So switching a profile in Compose does not turn off the corresponding service in Swarm — set the matching env var (DEPLOY_OPEA=0, ENABLE_OBSERVABILITY=0, CERTBOT_REPLICAS=0).

Topology catalog

Each topology below has: a name, a hardware sketch, the exact deploy command, what runs where, and a verification recipe. All commands assume the repo is cloned to /opt/genie-ai and the working directory is the repo root.

Topology A — Single-host (Compose dev, no GPU)

Use for: local development, quick smoke test, no AI workload.

flowchart TB
    host["localhost<br/>(no Swarm)"]
    services["all 18 always-on services<br/>(compose: no profiles;<br/>Swarm-equiv: DEPLOY_OPEA=0,<br/>ENABLE_OBSERVABILITY=0)"]
    host --> services

Set DEPLOY_OPEA=0 in .env to make this match the Swarm-equivalent behaviour (Compose ignores it; Swarm honours it).

Procedure: Docker Compose Setup → Step 6b.

Topology B — Single-host (Compose dev, with local GPU)

Use for: full local RAG with a developer GPU (T4 / RTX 6000 / A40).

flowchart TB
    host["localhost<br/>(no Swarm)"]
    services["all 18 always-on<br/>+ opea<br/>+ gpu-models"]
    host --> services

GPU model sizing: GPU Deployment.

Topology C — Single-node Swarm (all labels on one node)

Use for: small production on one host, dev clusters, CI smoke tests.

flowchart TB
    mgr["manager<br/>(gateway=true,<br/>genieai=true,<br/>gpu=true)"]
    svc["all 37 services<br/>(core + observability optional<br/>+ opea + gpu-models<br/>+ letsencrypt optional)"]
    mgr --> svc

# Init Swarm + label the manager for every role
docker swarm init
docker node update --label-add gateway=true  $(hostname)
docker node update --label-add genieai=true  $(hostname)
docker node update --label-add gpu=true     $(hostname)

# Auth to the registry (one-time)
docker login registry.opensource.unicc.org/un/itu/genie-ai

# Deploy
set -a && source .env && set +a
docker compose config > docker-compose.resolved.yaml
docker stack deploy -c docker-compose.resolved.yaml genieai

Verify:

docker service ls                       # all services 1/1 (healthy)
docker service logs genieai_kong-config | tail
# Look for: "Configuration restored successfully"
curl -sk https://localhost/api/health

Full procedure: Docker Swarm Setup → Step 13.

Topology D — Two-node Swarm (gateway+genieai on manager, gpu on worker)

Use for: small production where GPU memory pressure should not threaten the gateway or app node.

flowchart LR
    subgraph Manager["manager<br/>(gateway=true, genieai=true)"]
        mgr_svc["postgres, postgres-init<br/>kong*, nginx<br/>keycloak*, certbot<br/>frontend, redis-cache<br/>db-migrations, backend<br/>document-repository<br/>clamav, arango-vector-db<br/>(victoria*, tempo-proxy, grafana<br/>if obsv on)<br/>otel-collector (global)"]
    end

    subgraph Worker["worker<br/>(gpu=true)"]
        wrk_svc["vllm, vllm-translation-guardrail<br/>tei, tei_reranker<br/>textgen, embedding, reranker<br/>translation, guardrail<br/>dataprep-arango-service<br/>retriever-arango-service<br/>chatqna-xeon-* (disabled)"]
    end

# On manager
docker swarm init --advertise-addr <mgr-ip>
docker node update --label-add gateway=true <mgr>
docker node update --label-add genieai=true <mgr>

# On worker (after `docker swarm join --token ...`)
docker node update --label-add gpu=true <worker>

# On manager: deploy
set -a && source .env && set +a
docker compose config > docker-compose.resolved.yaml
docker stack deploy -c docker-compose.resolved.yaml genieai

Verify cross-node:

docker service ps genieai_backend  --format "{{.Node}} {{.Name}}"
docker service ps genieai_vllm     --format "{{.Node}} {{.Name}}"
# Both should show 1/1 running, each on the expected node.

Topology E — Three-node Swarm (production)

Use for: production with strict separation of concerns and predictable resource isolation.

flowchart LR
    subgraph Manager["manager<br/>(gateway=true)"]
        mgr_svc["postgres, postgres-init<br/>kong*, nginx<br/>keycloak*, certbot<br/>otel-collector (global)"]
    end

    subgraph W1["worker-1<br/>(genieai=true)"]
        w1_svc["frontend<br/>redis-cache<br/>db-migrations, backend<br/>document-repository<br/>clamav, arango-vector-db<br/>victoria*, tempo-proxy, grafana<br/>(if ENABLE_OBSERVABILITY=1)<br/>otel-collector (global)"]
    end

    subgraph W2["worker-2<br/>(gpu=true)"]
        w2_svc["vllm, vllm-translation-guardrail<br/>tei, tei_reranker<br/>textgen, embedding, reranker<br/>translation, guardrail<br/>dataprep-arango-service<br/>retriever-arango-service<br/>chatqna-xeon-* (disabled)"]
    end

# Manager
docker swarm init --advertise-addr <mgr-ip>
docker node update --label-add gateway=true <mgr>

# Worker-1
docker swarm join --token <token> <mgr-ip>:2377
docker node update --label-add genieai=true <worker-1>

# Worker-2
docker swarm join --token <token> <mgr-ip>:2377
docker node update --label-add gpu=true <worker-2>

# Manager: deploy
set -a && source .env && set +a
docker compose config > docker-compose.resolved.yaml
docker stack deploy -c docker-compose.resolved.yaml genieai

Verify placement:

for s in kong nginx keycloak backend vllm tei dataprep-arango-service retriever-arango-service grafana; do
  printf "%-30s %s\n" "$s" "$(docker service ps genieai_${s} --format '{{.Node}}' | head -1)"
done
# Expected: kong/nginx/keycloak → mgr ; backend/grafana → worker-1 ;
#           vllm/tei/dataprep/retriever → worker-2

Topology F — Remote GPU node (split app + AI)

Use for: when the GPU host is dedicated (bare-metal A40/H100, cloud GPU instance), and the app cluster runs elsewhere.

flowchart LR
    subgraph App["App node<br/>(Swarm, gateway+genieai+gpu)"]
        app_svc["all core + observability<br/>OPEA orchestrators<br/>(chatqna, retriever,<br/>dataprep, embedding wrapper,<br/>reranker wrapper, textgen,<br/>translation, guardrail)<br/>NO gpu-models profile containers"]
    end

    subgraph GPU["GPU node<br/>(standalone, no Swarm)"]
        nginx_g["nginx-gpu (TLS, port 443)"]
        llm["/llm/         → vLLM LLM"]
        tr["/translation/ → vLLM T"]
        em["/embed/       → TEI Emb"]
        rk["/rerank/      → TEI Rer"]
        dl["/docling/     → docling-serve"]
        nginx_g --> llm
        nginx_g --> tr
        nginx_g --> em
        nginx_g --> rk
        nginx_g --> dl
    end

    App -- "HTTPS (connect to remote GPU)" --> nginx_g

The app node must carry all three Swarm labels (gateway, genieai, gpu). The OPEA orchestrators above have placement: node.labels.gpu == true in docker-compose.yaml — without the gpu label they will not schedule. The gpu label on the app node only affects placement; the GPU-model containers (vllm/tei/tei_reranker/ vllm-translation-guardrail) stay down because their replicas are gated by GPU_MODEL_REPLICAS=0 (see below).

App node .env:

GPU_NODE_HOST=gpu.example.com
GPU_MODEL_REPLICAS=0
VLLM_API_KEY=<api-key-from-gpu-node-admin>
OPEA_SSL_SKIP_VERIFY=1     # only if GPU node uses self-signed certs

Deploy orchestrators only on the app node:

# App node — deploy Swarm stack
docker stack deploy -c docker-compose.resolved.yaml genieai
# GPU_MODEL_REPLICAS=0 skips vllm/tei/tei_reranker/vllm-translation-guardrail

GPU node: see Docker Swarm Setup → Remote GPU Node and deploy/ansible/README.md for the standalone 5-service AI stack.

Topology G — CPU-only Swarm (DEPLOY_OPEA=0)

Use for: stacks that need core services but no RAG pipeline (e.g. evaluation harnesses, keycloak-only test rigs).

# .env
DEPLOY_OPEA=0

Deploy normally:

docker stack deploy -c docker-compose.resolved.yaml genieai

With DEPLOY_OPEA=0, the OPEA orchestrator services (textgen, embedding, reranker, dataprep-arango-service, retriever-arango-service, chatqna-xeon-backend-server) start with replicas: 0 (their replicas are templated as ${DEPLOY_OPEA:-1}). translation cascades to replicas: 0 via ${GPU_MODEL_REPLICAS:-${DEPLOY_OPEA:-1}}. guardrail is hardcoded to replicas: 0 (always disabled — see the root CLAUDE.md port table). The chatqna-xeon-ui-server / chatqna-xeon-nginx-server are also at replicas: 0. The admin endpoints and chat UI still come up — chat responses will return abstain (no retrieval = no relevant docs).

Topology H — With observability (any topology A–G)

Use for: production where you want metrics, logs, traces, dashboards, and alerts.

Required additional vars:

ENABLE_OBSERVABILITY=1
GRAFANA_ADMIN_USER=admin
GRAFANA_ADMIN_PASSWORD=<strong-password>
KC_GRAFANA_CLIENT_ID=grafana
KC_GRAFANA_CLIENT_SECRET=<strong-secret>

Nine pre-built dashboards are auto-provisioned from configs/grafana/provisioning/dashboards/ — five application (application-metrics, rag-pipeline-trace-waterfall, service-health, service-logs, trace-explorer) and four infrastructure (observability-stack-health, victoriametrics-single-node, victorialogs-single-node, victoriatraces-single-node).

Access: https://<NGINX_PUBLIC_DOMAIN>/grafana/ (Keycloak OIDC login).

Note: victorialogs, otel-collector, and otel-collector-init start regardless of ENABLE_OBSERVABILITY because they have no profile and victorialogs runs with replicas: 1 while otel-collector runs in mode: global — the var only affects victoriametrics / victoriatraces / tempo-proxy / grafana. This is intentional: victorialogs is the admin-log destination even when full observability is disabled, and otel-collector collects container logs on every Swarm node via the fluentd driver.

Topology I — With Let’s Encrypt (any topology A–H)

Use for: production with a public FQDN and reachable port 80.

# .env
CERTBOT_EMAIL=your-email@example.com
NGINX_PUBLIC_DOMAIN=chat.example.com      # FQDN, not localhost

Certbot obtains, writes to secrets/ssl/, and renews every 12 hours; nginx reloads automatically. Set CERTBOT_STAGING=true to use the Let’s Encrypt staging server while testing (avoids rate limits).

Prerequisites: port 80 reachable from the internet, DNS A/AAAA record for the FQDN pointing to the host running certbot (the gateway node in Swarm).

Topology comparison table

Verify it worked (any topology)

After deploy, run this checklist against your topology:

# 1. Service list — all services should be 1/1 (or 0/N if disabled) and (healthy) or 'running'
docker service ls --format "table {{.Name}}\t{{.Replicas}}\t{{.Image}}\t{{.Ports}}" | head

# 2. Kong routes live — kong-config one-shot completes in ~10–30 s
docker service logs genieai_kong-config | tail
# Look for: "Configuration restored successfully"

# 3. Keycloak realm applied
docker service logs genieai_keycloak-config | tail

# 4. Smoke tests
curl -sk https://localhost/api/health           # backend
curl -sk https://localhost/                     # frontend HTML
curl -sk https://localhost/auth/realms/genie    # Keycloak realm info

# 5. AI stack (only if OPEA active)
docker service logs genieai_vllm | grep 'Application startup complete'
curl -sk https://localhost/api/health | python3 -m json.tool

# 6. Observability (only if H)
curl -sk https://localhost/grafana/api/health
docker service logs genieai_otel-collector | tail

# 7. GPU placement (topology D, E)
docker service ps genieai_vllm --format "{{.Node}} {{.Name}}"

Troubleshooting

I picked the wrong topology and a service won’t schedule

The service is pinned to a label the host does not have.

docker service ps genieai_<service> --no-trunc
# "no suitable node" → the target node lacks the required label.

docker node inspect <node> --format '{{.Spec.Labels}}'
# Add the missing label:
docker node update --label-add <label>=true <node>

Profile works in docker compose up but not in Swarm

Compose profiles are ignored by docker stack deploy. Set the matching env var instead:

Cross-node networking failing

Overlay network DNS can have first-startup delays. Check the overlay subnet and KONG_TRUSTED_IPS:

docker network inspect genieai_genieai_network --format '{{range .IPAM.Config}}{{.Subnet}}{{end}}'
# Swarm default is often 10.x.x.x — set KONG_TRUSTED_IPS=10.0.0.0/8

Remote GPU: OPEA services start but chat returns nothing

The orchestrators are up but cannot reach the remote GPU. Verify:

# 1. Override endpoints are set
grep -E '^VLLM_ENDPOINT|^EMBEDDING_SERVICE_URL|^RERANKER_SERVICE_URL|^VLLM_TRANSLATION_ENDPOINT|^DOCLING_ENDPOINT' .env

# 2. API key works
docker exec $(docker ps -q -f name=genieai_chatqna-xeon-backend-server) \
  curl -sk -H "Authorization: Bearer $VLLM_API_KEY" \
  https://${GPU_NODE_HOST}/llm/v1/models

# 3. OPEA services do not duplicate on the app node
docker service ps genieai_vllm --format "{{.DesiredState}} {{.Replicas}}"
# Should be 0/N — orchestrators only.

Observability: victorialogs (and the OTel Collector) run even though ENABLE_OBSERVABILITY=0

By design — victorialogs has replicas: 1 hard-coded and otel-collector runs in mode: global (one collector per Swarm node, no profile), so admin endpoints and the fluentd log pipeline keep working when full observability is disabled. To stop them, scale manually:

docker service scale genieai_victorialogs=0
docker service scale genieai_otel-collector=0   # stops it on every node

Certbot issued a certificate but nginx serves the self-signed one

Certbot writes to secrets/ssl/; nginx only reloads when the file changes. Either wait for the 12-hour renewal cycle, or force a reload:

docker exec $(docker ps -q -f name=genieai_nginx) nginx -s reload

Related cross-links

Procedures:

• Install Guide — end-to-end install (Ubuntu + Docker + Node.js + env config)
• Docker Compose Setup — docker compose up reference
• Docker Swarm Setup — docker stack deploy reference (covers topologies C, D, E, F)
• GPU Deployment — env.t4 / env.rtx6000 profiles, VRAM sizing
• NVIDIA A40 Install Guide — A40-specific driver walkthrough (topology C variant)

Architecture:

• Architecture Overview — C4 container view, service auth matrix
• Observability Overview — observability stack detail (topology H)