Deployment on GENIE.AI

Installation & Configuration Guide

Mon, 01 Jan 0001 00:00:00 +0000

This is the canonical end-to-end install procedure for GENIE.AI. Choose the deployment model that matches your case before reading on — each model has a dedicated sibling guide with the platform-specific mechanics, while this doc is the universal prerequisite + configuration + verification backbone:

Model	When to use it	Companion guide
Docker Compose (`docker compose up`)	Local dev, MVP, small single-host production, evaluation	Docker Compose Setup
Docker Swarm (`docker stack deploy`)	Production multi-node, HA, mixed CPU/GPU node placement	Docker Swarm Setup
Ansible + Swarm	Repeatable fleet deploys with per-environment secrets	Ansible README
Remote GPU (`GPU_NODE_HOST=`)	OPEA orchestrators on CPU node, vLLM/TEI on a separate GPU node	Remote GPU Node

GPU profiles (T4 16 GB / RTX 6000 24 GB / A40 48 GB / H100) layer on top of any of the above via the env.t4 / env.rtx6000 overrides — see GPU Deployment.

Docker Compose Setup

Mon, 01 Jan 0001 00:00:00 +0000

This guide covers local single-node deployment with docker compose up. For production multi-node deployments see Docker Swarm Setup.

GENIE.AI uses a single dual-mode docker-compose.yaml at the project root that works with both docker compose up and docker stack deploy. Compose is the fastest path to a working stack for development, testing, and small single-host production deployments.

Prerequisites

Docker Engine 23+ with Compose v2 (docker compose subcommand)
NVIDIA Container Toolkit (only if using --profile opea with GPU)
Hugging Face API token (only if using --profile opea — set HUGGING_FACE_HUB_TOKEN in .env)

For full system context, see Architecture Overview.

Docker Swarm Setup

Mon, 01 Jan 0001 00:00:00 +0000

This guide covers Docker Swarm deployments — single-node and multi-node. For single-host development with docker compose up, see Docker Compose Setup.

GENIE.AI uses a single Swarm-compatible docker-compose.yaml at the project root. All services deploy via docker stack deploy.

Prerequisites

Docker Engine 23+ with Compose v2 on every node
Same Docker version on all nodes (recommended)
NVIDIA Container Toolkit on GPU nodes — GENIE.AI exposes the GPU to containers via the NVIDIA_VISIBLE_DEVICES env var (default all), which requires nvidia-container-runtime registered as a Docker default runtime (or any runtime that supports NVIDIA_VISIBLE_DEVICES). Install on each GPU node: see https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html.
SSH access to every node — from the manager you need passwordless SSH to every worker (or docker -H ssh://worker ...) for cross-node debugging
Network connectivity between all nodes — Swarm requires TCP 2377 (cluster management), TCP+UDP 7946 (node-to-node gossip), UDP 4789 (overlay network VXLAN). Open these between every pair of nodes in the cluster.
Registry credentials — every node must authenticate to the GitLab Container Registry (docker login registry.opensource.unicc.org/un/itu/genie-ai) before docker stack deploy can pull images

For full system context, see Architecture Overview.

GPU Deployment

Mon, 01 Jan 0001 00:00:00 +0000

This guide covers how to deploy GENIE.AI’s AI layer on NVIDIA GPUs: choosing the right profile (T4 / RTX 6000 / A40), understanding the variables in env.t4 and env.rtx6000, sizing VRAM, and verifying the stack after deployment.

For full end-to-end installation see Install Guide (single-node) or Docker Swarm Setup (multi-node). This page is the GPU-specific layer on top of those.

Prerequisites

NVIDIA Driver installed on the GPU node. See NVIDIA Driver Installation Quickstart for the current matrix.
NVIDIA Container Toolkit installed and configured — see NVIDIA Container Toolkit overview. Verify with: docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi.
Docker Engine 23+ with Compose v2.
Swarm initialized if you plan to deploy via docker stack deploy: docker swarm init.
GPU node labeled: docker node update --label-add gpu=true $(hostname).
.env file created from the project env template (cp env .env) with secrets set.

Loading environment files

docker stack deploy does not support --env-file. The supported pattern is to source the files in the shell, resolve ${VAR} references into a flat YAML with docker compose config, then deploy the resolved file:

NVIDIA A40 Install Guide

Mon, 01 Jan 0001 00:00:00 +0000

This walkthrough installs the server-grade NVIDIA driver on Ubuntu 22.04 for an NVIDIA A40 GPU and resolves the common DKMS build failures caused by manually installed kernel headers. It is a pre-requisite for GENIE.AI on A40 hardware — once the driver is working, follow Docker Swarm Setup → Single-Node Swarm to deploy GENIE.AI.

Goal

Install the NVIDIA proprietary driver on Ubuntu 22.04 for an A40 GPU and bring up a single-node GENIE.AI Swarm stack on top of it, in ~30–60 minutes.

Deployment Topologies

Mon, 01 Jan 0001 00:00:00 +0000

This page is the single source of truth for which deployment topology fits which use case. It maps every supported combination of Docker Compose profiles, Swarm node labels, GPU presence, observability, and TLS to a named topology with a command, a diagram, and a verification recipe.

For the procedure to deploy a specific topology, follow the linked how-to guide. This page answers “which topology should I pick?” and “what runs where?”.