---

Architecture

graph TB
    subgraph clients["Clients"]
        dashboard["Dashboard · Next.js"]
        sdk["TypeScript SDK"]
        cli["CLI"]
    end

    subgraph platform["Xcelsior Platform"]
        api["FastAPI Gateway · public API + dashboard"]
        scheduler["Scheduler · Job matching & compliance"]
        db[("PostgreSQL 16 + SQLite auxiliaries")]
        billing["Billing · metering · holds · payouts"]
        reputation["Reputation · 6-tier trust scoring"]
    end

    subgraph hosts["GPU Host Network"]
        mesh["Headscale Mesh · Private Overlay"]
        worker1["Worker Agent · RTX 4090"]
        worker2["Worker Agent · A100 80 GB"]
        worker3["Worker Agent · H100 80 GB"]
    end

    dashboard & sdk & cli -->|HTTPS| api
    api --> scheduler
    api --> db
    api --> billing
    api --> reputation
    scheduler -->|SSH over mesh| mesh
    mesh --- worker1 & worker2 & worker3

---

Features

Global supply	Capacity from independent hosts worldwide, ranked on price, availability, hardware, and host reputation.
4-Layer Security	Version gating → least-privilege Docker → egress firewall → gVisor / Kata sandbox.
Agent-Native Surface	87 MCP tools with per-tool scopes — launch, connect, spend, snapshot, terminate. No dashboard step in any of it.
Metered Billing	Per-second metering against wallet holds that expire; Stripe, PayPal, and Wise payout rails; spending widened only through an approvable plan.
Reputation Engine	Multi-factor scoring with 7-day grace decay. Bronze → Silver → Gold → Platinum → Diamond tiers.
Private Mesh	Headscale overlay network — GPU workers never exposed to the public internet.
Admission Gating	Hosts must pass version checks + GPU fingerprinting before receiving any work.
Spot Instances	Interruptible GPUs at published spot rates (no bidding). Capacity-based preemption when on-demand needs capacity.

---

Spot Instances

Spot workloads use pricing_mode: "spot" — not an auction. Customers accept the published spot rate at launch; the rate is locked for billing when the job is assigned. Preemption is capacity-driven (on-demand contention or host drain), not bid-based.

# Launch an interruptible spot instance
curl -X POST https://xcelsior.ca/instance \
  -H "Authorization: Bearer $XCELSIOR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "embed-batch", "vram_needed_gb": 8, "pricing_mode": "spot", "gpu_model": "RTX 4090"}'

	On-Demand	Spot
Availability	Guaranteed while funded	Best-effort; may queue
Price	Fixed catalog rate	Published spot rate (typically 40–60% below on-demand)
Preemption	Never	Yes — when higher-priority work needs GPUs
Billing	Metered at on-demand rate	Metered at spot_rate_cad locked at allocation

Provider controls: spot_enabled, spot_gpu_slots, spot_min_cents per host. Kill switch: XCELSIOR_SPOT_ENABLED=false disables new spot launches (503). See SPOT_RUNBOOK.md (on-call) and SPOT_ROLLOUT.md (staging/production rollout).

Breaking change: max_bid and spot bidding are removed. Use pricing_mode=spot instead. See CHANGELOG.md.

---

Quick Start

git clone https://github.com/aabiro/xcelsior.git && cd xcelsior
uv sync && cp .env.example .env

# API
uvicorn api:app --reload --port 8000

# Worker (separate terminal)
XCELSIOR_HOST_ID=my-gpu XCELSIOR_SCHEDULER_URL=http://localhost:8000 python worker_agent.py

# Submit a job
python cli.py run my-model 8.0

Dashboard → http://localhost:8000/dashboard Health → http://localhost:8000/healthz Worker Auth → hosted/prod workers can use either XCELSIOR_API_TOKEN or XCELSIOR_OAUTH_CLIENT_ID + XCELSIOR_OAUTH_CLIENT_SECRET. If both are set, the worker prefers OAuth.

---

Host Admission Flow

sequenceDiagram
    participant H as GPU Host
    participant A as API Gateway
    participant S as Security Engine

    H->>A: PUT /host (gpu_model, vram, province)
    A-->>H: status = pending, admitted = false

    H->>A: POST /agent/versions (driver, runc, ctk, docker)
    A->>S: admit_node()
    S->>S: check_node_versions()
    S->>S: recommend_runtime()
    S-->>A: admitted = true | rejection reasons

    alt Admitted
        A-->>H: status = active → enters allocation pool
        loop Every 5s
            H->>A: POST /agent/telemetry
        end
    else Rejected
        A-->>H: status = pending, reasons logged
    end

Minimum versions — runc ≥ 1.1.14 · nvidia-ctk ≥ 1.17.8 · Docker ≥ 24.0 · Driver ≥ 535

---

Project Structure

api.py            FastAPI gateway (public API, SSE, dashboard)
scheduler.py      Job queue, host allocation, spot pricing, preemption
worker_agent.py   Pull-based GPU agent, telemetry, Docker execution
security.py       4-layer defense: version gating → gVisor/Kata
billing.py        metering, wallet holds, escrow, tax, payout rails
events.py         Append-only event store, tamper-evident hashing
reputation.py     Multi-factor scoring, Bronze→Diamond tiers
privacy.py        data retention, consent, export and erasure
verification.py   GPU fingerprint verification, re-verification scheduling
artifacts.py      Two-tier B2/R2 storage, policy-based routing
db.py             PostgreSQL ↔ SQLite dual-write, LISTEN/NOTIFY
cli.py            Full CLI for jobs, hosts, billing, marketplace
ai_assistant.py   Hexara AI assistant with tool-calling & onboarding wizards
routes/           Modular API route handlers (agent, instances, admin, etc.)
frontend/         Next.js 15 dashboard + marketing site (xcelsior.ca)
wizard/           Onboarding wizard: rent, provide, or SDK setup (TypeScript)
fern/             API documentation (docs.xcelsior.ca)
scripts/          Deployment, install, and bootstrap scripts
tests/            1400+ backend tests (pytest) + 17 frontend tests (vitest)

---

Deployment

Docker Compose (production)

docker compose up --build -d    # API + Frontend + Scheduler
curl https://xcelsior.ca/healthz

Systemd (alternative)

sudo cp xcelsior-*.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable --now xcelsior-api xcelsior-health

See .env.example for all 50+ configuration variables.

---

Testing

# Backend (1400+ tests)
uv run pytest tests/ -v                 # full suite
uv run pytest tests/ -v --cov=.         # with coverage

# Frontend (17 tests)
cd frontend && npm test                 # vitest

# Linting
ruff check . && black --check *.py routes/*.py tests/

CI runs automatically on push to main and on PRs — see CI and Frontend CI.

---

Contributing

1. Fork the repo
2. Create a feature branch (git checkout -b feat/my-feature)
3. Run the test suite — all tests must pass
4. Submit a PR with a clear description

See docs.xcelsior.ca for full API reference and integration guides.

---