Infrastructure¶

How the Inference Cooperative is built, for members who want to understand it — and contributors who want to work on it.

The big picture¶

Members reach the co-op through two front doors: the chat (LibreChat) and the API (the LiteLLM gateway). Both route through a single gateway to multiple LLM providers — so members choose what they value for each task: architecturally private inference in TEE-protected enclaves (Tinfoil), 100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware — the member portal — ties Open Collective membership to Cloudron accounts and LiteLLM keys.

                                              ┌─> Tinfoil (TEE)      🔒 private
Members ──> LibreChat (chat) ──┐              │
                               ├──> LiteLLM ──┼─> GreenPT (renewable) 🌱 green
Members ──> your own tools ────┘    gateway   │
                          (API)     (keys,    └─> Regolo (renewable)  🌱 green
                                    metering,
                                    routing)

The whole stack runs on Cloudron, which handles hosting, single sign-on, and per-app isolation.

Where it runs¶

The co-op's services run on a server the co-op owns and pays for directly, at Infomaniak — a Swiss sovereign cloud provider whose data centres are in Geneva, run on renewable energy, and certified to ISO 14001, ISO 50001 and ISO 27001. The stack moved there from a personal Hostinger account on 2026-11-02; nothing about how members use the co-op changed in the move.

One thing runs elsewhere: the co-op's steward bot — the automation behind the digests and the newsletter drafts — runs on a separate machine, not on the Cloudron server, and was not part of the migration.

DNS¶

inference.coop is managed by hand at domains.coop (nameservers ns1/ns2.yoursrs.com). The zone is deliberately small — eleven records. Most of the co-op hangs off just a few of them: the apex (inference.coop), www, and a wildcard (*.inference.coop) point at the server, so each application hostname — chat, portal, dashboard, gateway, git, newsletter, vault and the rest — is served by the wildcard rather than having a record of its own. The DNS change sheet listing all eleven records is public, in co-op/runbooks.

Backups¶

Cloudron writes encrypted backups to Infomaniak's object storage (the "Infomaniak Encrypted S3" site, in Switzerland) every night, and keeps the last seven days. The steward-bot host backs up its own selected home directories to the same object storage.

Components¶

Component	Purpose	URL	Code
LibreChat	Member-facing chat (models, web search, image generation, voice)	chat.inference.coop	co-op/librechat-custom
Open WebUI	Legacy chat instance — read-only reference (old history + file attachments)	chat-legacy.inference.coop	—
LiteLLM	AI gateway — keys, metering, model routing	gateway.inference.coop	code/litellm
Member Portal	Membership middleware — onboarding, key provisioning, Loomio sync	portal.inference.coop	code/member-portal
Member Dashboard	Member-facing usage view + API key management	dashboard.inference.coop	code/member-dashboard
Admin Panel	Admin-only member overview, spend, balance management	panel.inference.coop	code/admin-panel
SearXNG	Self-hosted web search (feeds the chat's search)	search.inference.coop	—
Loomio	Member governance	forum.inference.coop	—
Matrix (Synapse)	Member chat server (federated; SSO login via Cloudron)	matrix.inference.coop	—
Gitea	Git hosting (code + docs)	git.inference.coop	—
Surfer	Static hosting (landing page, thanks page)	inference.coop	co-op/website
Listmonk	Member newsletter (auto-synced from membership)	newsletter.inference.coop	—
Vaultwarden	Password manager (admin credentials)	vault.inference.coop	—
SnappyMail	Webmail	email.inference.coop	—

Open Collective (billing and fiscal sponsorship) is the one external service in the flow — see opencollective.com/inference-cooperative.

Mail¶

The inference.coop mail domain is the co-op's, and the mail itself runs on the co-op's Cloudron server.

Incoming mail is received directly by that server — the domain's MX record points at it, it accepts mail on the standard SMTP port, and webmail and mail apps reach it over the usual encrypted ports (IMAP and SMTP over TLS). If mail to your @inference.coop address stops arriving, that is a fault on our side to fix, not a limitation of how the co-op is set up.

Outgoing mail travels by two different paths, on purpose:

The newsletter (Listmonk) sends through Scaleway Transactional Email, an EU-based SMTP relay, rather than from our own server. Our domain is authenticated to that relay with its own DKIM key, and a real message has been checked end to end: SPF, DKIM and DMARC all pass under a p=reject policy — so anything claiming to be from us that the relay hasn't authenticated is rejected outright rather than quietly filed away.
The member portal's transactional email — welcome messages, budget notices, the confirmation the nightly membership sweep sends — still sends from the co-op's own server, through the mail service Cloudron runs for us.

Both paths send as Inference Cooperative <info@inference.coop>.

Why split them: sending the newsletter through a relay lets a provider with established sending IPs carry a list this size, instead of leaning on one freshly-migrated server's reputation. Receiving we keep in-house, on our own hardware.

Matrix¶

Members get a federated Matrix account on our self-hosted Synapse (matrix.inference.coop, user IDs @username:inference.coop) using their Cloudron SSO login. The portal invites every active member — and Nathan's federated account — to the private Inference Cooperative Space (with a General room) on every membership sync, so new members land in the space automatically. Invites honor rate limits; external/federated members can be added via the portal's MATRIX_EXTRA_MXIDS env.

Federation is enabled (delegated via .well-known/matrix/server on the bare domain, port 443), so members can also join from any homeserver they already use.

Which account is which¶

Several automated accounts touch this homeserver, and two of them once differed by a single character — which caused real confusion, including a bot appearing to announce itself. The distinction is deliberate:

@inferencebot:inference.coop — the steward. The bot members actually meet: it posts digests, newsletter notes and reminders. It is not a Synapse server admin, and holds no room power beyond what it needs to post.
@inference-admin:inference.coop — the room and server administrator. A Synapse server admin, used for provisioning rooms, managing membership, and retiring rooms. It never speaks in a member room; a message from it is a mistake, not a feature.
@infrastructure-bot:inference.coop — the infrastructure working group's bot. Answers infrastructure questions in #infrastructure when mentioned.
@ops_audit:inference.coop — a server-admin account created for the October 2026 security review. Retained only until its tokens are revoked.

House rule: power belongs to @inference-admin, voice belongs to @inferencebot. Please don't create another account whose name differs from an existing one by a character or two; that is the specific confusion this section exists to undo.

@inference-bot:inference.coop is retired (2026-10-29). It was the original room-administration account (it created the space and most of the rooms) and held a Synapse server-admin flag. Its room power went to @inference-admin, which is now administrator of record for #general, #community, #design, #skillshare, #updates and

infrastructure, and holds the portal's member-invite credential¶

(MATRIX_BOT_TOKEN) under its own token. The old account's admin flag was dropped, it was deactivated, and every device and token was deleted. Nothing should use it again.

Two loose ends, recorded so nobody re-investigates them from scratch: it remains joined to two rooms it cannot leave (they reject every write — 500 on leave), and those rooms are also why they can't be deleted; and the old account's name is still the tempting one, so check whoami rather than the name before trusting any account that looks like it.

How the apps run themselves¶

The portal and gateway are self-maintaining: their recurring housekeeping is built into the apps as scheduled tasks, not run from any member's or operator's personal machine.

Member portal — two scheduled jobs (Cloudron scheduler):
Membership sweep (nightly, 3am) — reconciles members against the live Open Collective list: provisions new members who signed up while a webhook was missed, deactivates lapsed members, and syncs Loomio + Matrix membership. A missed webhook self-heals within 24 hours.
Docs knowledge refresh (nightly, 3:30am) — pulls this documentation from the public Git repo and updates the shared knowledge base behind the chat's models, so answers grounded in the docs stay current. Changes to the docs appear in the chat within a day, no manual step.
Gateway — an audio keep-alive (every 10 minutes) sends a tiny text-to- speech ping to the private enclave so voice mode responds quickly even after long quiet periods, without compromising the end-to-end encryption model.

Scheduled tasks are declared in each app's CloudronManifest.json (see the code/ repos) and survive redeploys. If a task fails, it shows up in the app's logs on the admin panel; the portal also exposes a manual trigger for the membership sweep.

How membership is wired¶

Membership flows through the member portal, which is the single source of truth for "who is a member":

A member contributes on Open Collective.
Open Collective fires a webhook to the portal, which provisions a Cloudron account, a LiteLLM key, and a welcome email.
A nightly sweep reconciles against the live Open Collective list, so a missed webhook self-heals within 24 hours.

The portal database is authoritative for member emails and balances; LiteLLM's team budget is the source of truth for a member's allowance, shared between chat and API.

The bot account on Open Collective¶

You may notice "Inference Co-op Bot" listed as an admin of the collective on Open Collective. This is infrastructure, not a member or a person:

What it is: a dedicated Open Collective account owned by the co-op. It holds a Personal Token that the member portal uses to read membership data — the roster, tiers (needed to route credit-pack purchases correctly), and contributor emails (including guests, which only admins can see).
What it can do: read member and tier data via Open Collective's API. It cannot move money, issue refunds, or change the collective — Open Collective restricts those to dashboard actions by human admins. Its API access was verified read-mostly during the October 2026 security review.
Why it has admin: reading guest contributor emails requires admin on Open Collective; without it, membership reconciliation and credit-pack routing can't function.
Who supervises it: all of its code is open source in code/member-portal — every API call it makes is documented there. Nathan Schneider (collective admin) oversees its use and can revoke or rotate its token at any time from the Open Collective dashboard.

Nothing on Open Collective is charged, transferred, or refunded by automation: payments always come from a human's own contribution action, and any automated system only reads the resulting records.

The code¶

The cooperative's software is open source, in the code organization on Gitea:

code/member-portal — membership middleware (Python).
code/member-dashboard — member-facing usage + API keys (Python).
code/admin-panel — admin overview (Python).
code/litellm — the gateway packaged as a Cloudron app.

Plus the co-op organization:

co-op/website — landing page and static site.
co-op/docs — this documentation.
co-op/design-assets — logos, fonts, brand materials.

Contributing (the Git is yours, too)¶

Everything the co-op runs is open source and lives at git.inference.coop — the website, this documentation, the member portal, the dashboard, the admin panel, and the gateway. As a member, you can browse all of it and propose changes.

Who can sign in. Git access is tied to your membership: you log into the Git with the same single sign-on you use for the chat. Members are in automatically (through the members group); when a membership lapses, Git access lapses with it.

How to contribute. Anyone can read the repos without an account. To make a change, fork the repo, edit in your fork, and open a pull request:

Log into git.inference.coop with your co-op account.
Fork the repo you want to change (e.g. co-op/docs).
Make your edits in your fork and commit them.
Open a pull request back to the original repo, describing your change.

All changes are reviewed. Nothing merges directly into main — every change comes in as a pull request that an admin reviews and approves first. You don't need to be able to create repos or organizations to contribute; you just fork, edit, and ask for review.

Questions about the Git itself? Write to info@inference.coop.

Models and providers¶

The co-op serves models from multiple providers (Tinfoil for privacy, GreenPT for renewable energy, PublicAI coming soon). See the model list for the current catalog — that's the single source of truth for which models are available and how they're named.

One infra detail worth knowing: a local Tinfoil proxy sidecar handles the enclave encryption on our side of the gateway — only Tinfoil traffic goes through it; the other providers are called directly.

Security posture (plain language)¶

Token-gated control plane. The portal is publicly reachable, but every sensitive surface requires a secret token (admin, broker, webhook). All fail closed — unauthenticated requests get 401.
Separated secrets. Chat, broker, admin, and model-layer secrets are independent and independently rotatable, so a leak has bounded scope.
Member isolation. Each API key is scoped to its owner's LiteLLM team and budget; a member can only ever act on their own keys.
Rate limiting on control-plane endpoints; the inference path is capped by each member's budget instead.
No secrets in version control. Credentials live in Cloudron env vars, never in the repos.
Accepted trade-offs (documented, not hidden): member keys are stored in plaintext in the portal database (functionally necessary), and the gateway must stay publicly reachable for API access (its admin UI and docs are disabled; only the /v1/* API is exposed).

For operators¶

Maintaining the co-op's infrastructure — rotating Cloudron tokens, editing the portal's environment, and troubleshooting the provisioning pipeline — is documented in the operator runbook.

Incident log¶

Write-ups for outages live in incidents/. One so far: 2026-10-02 — gateway outage after a security-upgrade rebuild.