Skip to content

System architecture

This page looks inside each component and at the contracts between them. Each contract has an owner, and that owner documents it in its own repository:

Contract Transport Owner Reference
Station → host results and live data MQTT (JSON) nestris-core (docs/STATION.md) MQTT contract
Station → host game recording HTTP PUT …/ngf nestris-core / NestrisLTM NGF recordings
Terminal ↔ host REST, bearer token NestrisLTM Terminal API
Reader ↔ station / terminal USB serial, JSON lines, protocol v2 nestris-rfid-reader (docs/PROTOCOL.md) Reader protocol
Player card layout MIFARE Classic sector 1 nestris-rfid-reader (docs/CARD_FORMAT.md) Card format

NestrisLTM (host)

NestrisLTM is one Python process with two halves. A Qt shell runs on the main thread. A background asyncio thread runs the web server, the MQTT ingest and the workers. With --headless the same core runs without Qt, for example on a Linux server.

flowchart LR
    subgraph EXT["Outside the process"]
        MQ["<span style='color:#ffffff'>Mosquitto<br/>:1883</span>"]:::ext
        PG[("<span style='color:#ffffff'>PostgreSQL<br/>:5432</span>")]:::ext
        BR["<span style='color:#ffffff'>Browsers · OBS ·<br/>terminal · stations</span>"]:::ext
        GH["<span style='color:#ffffff'>GitHub Releases</span>"]:::ext
    end

    subgraph PROC["NestrisLTM process"]
        direction TB
        subgraph QT["Main thread · PySide6"]
            TRAY["<span style='color:#111111'>Tray icon<br/>health colours</span>"]:::ltm
            WIN["<span style='color:#111111'>Window with<br/>embedded browser</span>"]:::ltm
        end
        subgraph CORE["Background thread · asyncio"]
            ING["<span style='color:#111111'>MQTT ingest<br/>(aiomqtt, persistent session)</span>"]:::ltm
            HUB["<span style='color:#111111'>LiveHub<br/>in-memory live state</span>"]:::ltm
            SPOOL["<span style='color:#111111'>Disk spool +<br/>ingest worker</span>"]:::ltm
            FR["<span style='color:#111111'>Frame writer<br/>(batched 1 s)</span>"]:::ltm
            API["<span style='color:#111111'>FastAPI / uvicorn<br/>:7990 REST + WebSocket</span>"]:::ltm
            WS["<span style='color:#111111'>WebSocket hub</span>"]:::ltm
            UPD["<span style='color:#111111'>Update checker</span>"]:::ltm
        end
    end

    MQ -- "status · player · live" --> ING --> HUB
    ING -- "event/*" --> SPOOL --> PG
    HUB --> FR --> PG
    HUB --> WS
    PG <--> API
    WS --> API
    API <--> BR
    API -- "cmd → reader" --> MQ
    UPD --> GH
    TRAY & WIN -.-> API

    classDef ltm fill:#f5b800,stroke:#7a5c00,color:#111,stroke-width:2px
    classDef ext fill:#64748b,stroke:#334155,color:#fff,stroke-width:2px

Key properties

  • Durable ingest. Game events (game_start, cheat, game_end) are written to a disk spool before they touch the database. Writes are idempotent, keyed by the station's game_id, so duplicate deliveries are harmless.
  • Live path. live frames (up to 60 Hz per station) go to the in-memory LiveHub and from there straight to the WebSocket clients. They are also stored once per second until the full recording arrives from the station and replaces them.
  • Self-migrating database. The nestrisltm database is created automatically, and Alembic migrations run on every start.
  • Events define the view. Each event has a time window and exactly one event is active. Highscore, bracket and overlays always show the active event.
  • Security model. The admin UI needs a login, and every write API needs a scoped token (admin, players:write, games:write, stations, scenes, terminal). Overlays and displays are read-only and open on the LAN.

Frontend. The admin UI and the overlays are Svelte 5 apps. A shared package renders NES playfields and decodes NGF recordings. The same package is used by the player terminal.

nestris-core (recognition engine and station)

nestris-core is a Rust workspace. It is a high-performance port of the original Python recognition engine and is wire-compatible with it. The engine uses classical computer vision, with no machine learning:

flowchart LR
    SRC["<span style='color:#f8fafc'>Video source<br/>capture card · file · stream</span>"]:::hw --> FF["<span style='color:#111111'>ffmpeg<br/>decode → 1280×720</span>"]:::core
    FF --> LOCK["<span style='color:#111111'>Geometry lock<br/>playfield + label anchors<br/>RANSAC homography</span>"]:::core
    LOCK --> WARP["<span style='color:#111111'>Warp to canonical<br/>NES frame 256×240</span>"]:::core
    WARP --> REC["<span style='color:#111111'>Recognition<br/>score · lines · level · next ·<br/>playfield · piece stats</span>"]:::core
    REC --> FUS["<span style='color:#111111'>State fusion<br/>plausibility · events</span>"]:::core
    FUS --> STATS["<span style='color:#111111'>Statistics<br/>tetris rate · burn · drought · pace</span>"]:::core
    STATS --> SINK["<span style='color:#111111'>Sinks</span>"]:::core
    SINK --> J["<span style='color:#ffffff'>JSONL / NDJSON /<br/>WebSocket</span>"]:::ext
    SINK --> NGF["<span style='color:#ffffff'>NGF recording<br/>.ngf.gz</span>"]:::ext
    SINK --> MQTT["<span style='color:#ffffff'>MQTT<br/>(station only)</span>"]:::ext

    classDef core fill:#22d3ee,stroke:#0e7490,color:#111,stroke-width:2px
    classDef ext fill:#64748b,stroke:#334155,color:#fff,stroke-width:2px
    classDef hw fill:#1f2937,stroke:#94a3b8,color:#f8fafc,stroke-dasharray:4 3

The geometry lock is a small state machine. It finds the playfield on its own and re-solves in the background when the picture drifts:

stateDiagram-v2
    direction LR
    [*] --> UNLOCKED
    UNLOCKED --> ACQUIRING: candidate found
    ACQUIRING --> LOCKED: homography confirmed
    ACQUIRING --> UNLOCKED: rejected
    LOCKED --> DRIFT: picture moved
    DRIFT --> LOCKED: re-solved
    DRIFT --> LOST: no fit
    LOST --> ACQUIRING: search again

Crate structure

flowchart BT
    V["<span style='color:#111111'>nestris-vision<br/><small>CV primitives, no I/O</small></span>"]:::core
    E["<span style='color:#111111'>nestris-engine<br/><small>recognition, fusion, stats, integrity</small></span>"]:::core
    N["<span style='color:#111111'>nestris-ngf<br/><small>NGF codec, recorder, replay</small></span>"]:::core
    H["<span style='color:#111111'>nestris-host<br/><small>ffmpeg capture, sinks</small></span>"]:::core
    GC["<span style='color:#111111'>nestris-gui-core</span>"]:::core
    CLI["<span style='color:#111111'>nestris-cli<br/><small>nestris binary</small></span>"]:::core
    GUI["<span style='color:#111111'>nestris-gui<br/><small>egui desktop</small></span>"]:::core
    QT["<span style='color:#111111'>nestris-qt-gui<br/><small>Qt 6 / QML</small></span>"]:::core
    WASM["<span style='color:#111111'>nestris-wasm<br/><small>browser GUI</small></span>"]:::core
    STN["<span style='color:#111111'>nestris-station<br/><small>tournament daemon</small></span>"]:::core

    E --> V
    N --> E
    H --> N
    GC --> H
    CLI --> H
    GUI --> GC
    QT --> GC
    WASM --> N
    STN --> H

    classDef core fill:#22d3ee,stroke:#0e7490,color:#111,stroke-width:2px

nestris-station is the headless daemon that runs on every tournament station. It is a small Debian PC with a USB capture stick and an RFID reader:

  • It attributes each game to the card on the reader.
  • It detects the Select score trick (+10 000 points) and validates every result.
  • It publishes live state and results over MQTT. Durable messages are spooled until the broker acknowledges them.
  • It records every game as .ngf.gz and uploads it to NestrisLTM.
  • It runs as a systemd service with a watchdog, restarts capture with backoff and reconnects the reader and broker by itself.

nestris-terminal (registration kiosk)

The terminal never talks to the database. It is a client of NestrisLTM's REST API, and the token stays inside a local bridge process, so the touch UI never sees it.

flowchart LR
    subgraph PC["Touch PC"]
        direction TB
        KIOSK["<span style='color:#111111'>Kiosk window<br/>PySide6 + QWebEngine<br/>full screen</span>"]:::term
        UI["<span style='color:#111111'>Svelte touch UI</span>"]:::term
        subgraph BRIDGE["Local bridge · FastAPI 127.0.0.1:7991"]
            direction TB
            PX["<span style='color:#111111'>/api/* proxy</span>"]:::term
            LOC["<span style='color:#111111'>/local/* card write · admin</span>"]:::term
            WSB["<span style='color:#111111'>/ws events for UI</span>"]:::term
            DRV["<span style='color:#111111'>RFID driver<br/>(serial / fake)</span>"]:::term
            CL["<span style='color:#111111'>Host client<br/>bearer token</span>"]:::term
        end
        KIOSK --> UI
        UI <--> PX & LOC & WSB
        PX --> CL
        LOC --> DRV
        DRV --> WSB
    end
    RD["<span style='color:#111111'>RFID reader</span>"]:::rfid <-- "USB serial · protocol v2" --> DRV
    CL -- "HTTP · /api/terminal/v1" --> LTM["<span style='color:#111111'>NestrisLTM :7990</span>"]:::ltm

    classDef ltm fill:#f5b800,stroke:#7a5c00,color:#111,stroke-width:2px
    classDef term fill:#a78bfa,stroke:#6d28d9,color:#111,stroke-width:2px
    classDef rfid fill:#f472b6,stroke:#be185d,color:#111,stroke-width:2px

nestris-rfid-reader (firmware)

The reader is deliberately simple. It has no Wi-Fi and talks only USB serial. It reports card changes within about 300 ms, shows text from the host on its OLED, and writes names to cards safely (write, read back, verify).

flowchart LR
    CARD["<span style='color:#f8fafc'>MIFARE Classic<br/>player card</span>"]:::hw -. "13.56 MHz" .-> RC["<span style='color:#111111'>RC522<br/>(SPI)</span>"]:::rfid
    subgraph ESP["ESP32 DevKit"]
        direction TB
        FW["<span style='color:#111111'>Firmware<br/>card tracker · protocol v2 ·<br/>watchdog · NVS settings</span>"]:::rfid
    end
    RC --> FW
    FW --> OLED["<span style='color:#111111'>SSD1306 OLED<br/>(I²C) 128×32 / 128×64</span>"]:::rfid
    FW <-- "USB serial 115200 · JSON lines" --> HOSTAPP["<span style='color:#ffffff'>nestris-station or<br/>nestris-terminal</span>"]:::ext

    classDef rfid fill:#f472b6,stroke:#be185d,color:#111,stroke-width:2px
    classDef ext fill:#64748b,stroke:#334155,color:#fff,stroke-width:2px
    classDef hw fill:#1f2937,stroke:#94a3b8,color:#f8fafc,stroke-dasharray:4 3

Technology stack

Layer NestrisLTM nestris-core nestris-terminal nestris-rfid-reader
Language Python 3.13 Rust (edition 2024) Python 3.13 C++ (Arduino)
Runtime / framework FastAPI, SQLAlchemy, Alembic, aiomqtt, PySide6 ffmpeg, egui, Qt 6, wasm-bindgen FastAPI, httpx, pyserial, PySide6 PlatformIO, ESP32 Arduino core
UI Svelte 5 egui / QML / TypeScript + wasm Svelte 5 SSD1306 OLED
Storage PostgreSQL 14+ NGF files, on-disk spool TOML config ESP32 NVS
Packaging PyInstaller + Inno Setup .deb, zip bundles, AppImage, .app PyInstaller + Inno Setup .bin images
CI / release GitHub Actions, Ed25519-signed GitHub Actions, Ed25519-signed GitHub Actions, Ed25519-signed GitHub Actions, Ed25519-signed