Synced from Hive. This page is pulled from kubestellar/hive@v4 during the docs build. Edit the canonical source in the Hive repository.

Hive Reference Architecture

Hive is AI-agent orchestration for open-source projects. A single Go binary enumerates GitHub issues and PRs, classifies them, and dispatches work to AI coding agents (Claude, Copilot, Gemini, Goose) on adaptive cadences governed by queue depth — with layered, technically-enforced guardrails between an agent and the outside world.

The design separates deterministic decisions (filtering, classification, merge-gating, permission enforcement) from judgment calls (reading code, reasoning about a fix, writing a PR). Deterministic work runs in Go and shell before any LLM sees the task; agents handle the judgment.

Goal decomposition, plan review, and stall-triggered re-planning are not covered on this page — they are the planning-intelligence lane. Plan review and the stall-replan lane run automatically (replan is on by default); decomposition is gated at ACMM L5+, because the architect that decomposes epics has no cadence below L5, and its final step — turning the architect’s plan into child beads — currently runs through the bd decompose CLI rather than automatically. See planning-intelligence.md and ADR-0006.

1. System context

Where Hive sits between the humans who run it and the services it drives.

2. Container process model

Hive ships as container. The Docker entrypoint is PID 1 and supervises three long-lived processes; the Go binary is the brain and runs most subsystems as in-process goroutines.

Agents run as tmux sessions named hive-<agent>, per agent, optionally under per-agent OS users for UID isolation.

3. The governor loop — from queue depth to a kick

The governor is a queue-depth scheduler. Each eval cycle it enumerates actionable GitHub work, picks a mode from the issue count, derives each agent’s cadence from that mode, and returns the agents that are due — which the scheduler turns into kick messages the agent manager types into each tmux session.

Modes (thresholds are config/pack-driven; defaults shown):

A kick is a work order typed into the agent’s CLI prompt. If the agent has clear_on_kick, the manager sends /clear first so each kick starts from a clean context. A 7-day rolling token budget gates kicks: at 90% it warns; on exhaustion it suppresses kicks for all but explicitly-exempt agents.

4. The deterministic pipeline

Before any agent is kicked, run-pipeline.sh runs the pre-kick stages defined in hive-project.yaml, topologically sorted by their declared dependencies. Each stage is a shell script that writes a JSON artifact other stages and agents consume — so an agent is handed pre-filtered, pre-classified, merge-gated work. For a script-by-script index of this layer, see ../../bin/README.md.

• Classification (also mirrored in Go, pkg/classify) tags each issue with a complexity tier (Simple/Medium/Complex → haiku/sonnet/opus), a lane (which agent owns it), and a cluster key for bundling related work.
• Merge-gating produces merge-eligible.json; agents may merge only PRs on that list (required CI green — ignoring Playwright/Tide/visual checks — not draft, and either AI-authored or community-authored with an approving review).
• Enforcement is the always-on gh wrapper (/usr/local/bin/gh): it injects the scoped App token and blocks writes that exceed the agent’s permission tier — the shell-level twin of the network-level MITM proxy (§6).

5. Layered guardrails (defense in depth)

An agent’s permissions are enforced at three independent layers, all keyed off the same per-agent mode (ADVISORY → ISSUES_ONLY → ISSUES_AND_PRS → ISSUES_PRS_MERGE) that the ACMM level assigns. A bug in layer is caught by the next.

MITM proxy rule table (first match wins; a write below its MinMode gets a 403 X-Hive-Proxy-Blocked):

Agent identity is derived from the connection’s owning UID (/proc/net/tcp → UID → agent name), and the agent’s mode is read from a hot-reloadable file /tmp/.hive-mode-<agent>. A repo allowlist additionally blocks writes to any repo outside the configured set. api.github.com is inspected; github.com (OAuth, git smart-HTTP) is tunneled opaquely.

6. ACMM — controlling agent autonomy

The AI-native Capability Maturity Model is the single dial an operator turns. The level maps deterministically to per-agent modes via DefaultAgentMode, which in turn sets the guardrails in §5. Raising the level is always a human decision.

supervisor is always advisory. The full matrix is in acmm-policy-matrix.md.

7. Beads — the work ledger

Each agent has a durable, git-backed JSON ledger (bd CLI) that lets agents coordinate without a central queue. Beads are typed work items with priorities, dependencies, and free-form metadata; agents scope their view with --actor and pull ready work with bd ready.

• Types: bug · feature · task · epic · chore · decision · advisory
• Status: open · in_progress · blocked · done · closed
• Location: /data/beads/<agent>/beads.json (+ archived archive.jsonl)
• Blocked writes and other findings land as advisory beads that the governor folds into its digest — the ledger is internal state, never mirrored to GitHub.

8. Hub & spoke

Every hive is a spoke; hosted instance (hive.kubestellar.io) runs as the hub. Both are the same image (HIVE_MODE=hub selects the role). Spokes push a heartbeat; the hub answers with callbacks — the control channel that works even for spokes the hub can’t reach directly (firewalled clusters).

The hub also serves the public registry, cross-hive leaderboard, and the contributor flow: community members donate compute to a spoke via ClankeR, the contributor relay — starting rate-limited and auto-promoting through trust tiers as tasks complete — their credentials never leave their machine.

9. Model backends & cost

Agents can run against Anthropic (Claude Code), GitHub Copilot, or any OpenAI-compatible gateway (vLLM, llm-d, LiteLLM, named gateways). Gateway traffic is routed through the in-process inference translator (:18444), which reroutes Anthropic-shaped calls to OpenAI-shaped endpoints where needed.

Cost is a list-price estimate, not a billing feed: the token collector scans each backend’s session JSONL files (plus a live proxy sniff of Copilot’s usage block) and multiplies token counts by a dated per-model price table. This feeds the dashboard’s live cost, hourly spend, and per-agent/model attribution. See Token collection and usage tracking for the data shape, /api/cost, and hub /api/saas/usage rollups.

10. Dashboard & observability

• /api/status — full fleet + governor state (BuildFrontendStatus).
• /api/audit — recent audit entries for read-write users: dashboard config changes, logins, GitHub App setup changes, and agent lifecycle events such as start, stop, launch failure, pause/resume/kick, add/remove, backend changes, and model changes. Entries are kept in memory and, when /data exists, appended to /data/audit.jsonl with lumberjack rotation (5 MB files, 3 backups, 90 days).
• The machine-readable dashboard API reference is dashboard/openapi.json.
• /api/events — Server-Sent Events; the dashboard is pushed a fresh snapshot on every eval cycle (and a lighter agent-only update on the fast poll).
• /api/health, /api/health/deep, /api/livez — readiness and liveness; the livez probe catches the “HTTP up but eval loop / heartbeat stalled” case.
• Notifications (ntfy / Slack / Discord) fire on budget warnings, SLA breaches, trajectory-drift pauses, and other governor events.
• Optional OpenTelemetry export is configured with an otel: block in hive.yaml and is off by default. When enabled, Hive exports OTLP/HTTP spans for governor eval cycles, agent kicks, and recorded lifecycle/PR events; agent spans use GenAI semantic convention attributes such as gen_ai.system, gen_ai.request.model, and token usage fields when that data is available, plus Hive attributes like hive.agent, hive.lane, hive.acmm_level, and hive.governor.mode.
• Log output is wrapped by pkg/logscrub, which redacts recognized GitHub token and JWT-like strings from messages and string attributes. See Security notes for guarantees and limits.
• Network exposure and TLS termination are documented in Network and port requirements and TLS setup.
• /terminal is served by ttyd, which invokes deploy/ttyd-tmux.sh to attach to the selected hive-<agent> tmux session as the socket-owning UID. This is required when agents run under per-agent users and tmux rejects attaches from the proxy user.
• deploy/hive-panes.sh backs a read-only peer-observation workflow: it reads pluk JSONL logs from /var/run/pluk/logs, skips the calling agent, strips terminal escapes, and prints recent output without opening another agent’s tmux socket. See Agent peer-awareness logging for the log format, attachment conditions, and retention.

11. End-to-end: an issue becomes a merged PR

Putting the pieces together for a single unit of work at L6.

Data stores at a glance

See also: docs index · agent-configuration.md · acmm-policy-matrix.md · security-threat-model.md · ADR index · roadmap.md · landscape.md · manual-provisioning.md · cross-cluster-migration.md · trajectory-review.md · design/knowledge-system.md