From 4564aa53cc8a2ffca152d669ceae928e6f4a2bbd Mon Sep 17 00:00:00 2001 From: dhkatz Date: Thu, 29 Oct 2026 02:01:08 -0700 Subject: [PATCH] Design draft for the hub, catalogs and community products Co-Authored-By: Claude Opus 5.5 --- .gitattributes | 1 + CLAUDE.md | 13 +++ README.md | 6 ++ docs/design.md | 284 +++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 304 insertions(+) create mode 100644 .gitattributes create mode 100644 CLAUDE.md create mode 100644 README.md create mode 100644 docs/design.md diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..6313b56 --- /dev/null +++ b/.gitattributes @@ -0,0 +1 @@ +* text=auto eol=lf diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..d6ed46b --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,13 @@ +# t5de-hub + +T5DE's hub (see docs/design.md, a draft under discussion with the user). Related repositories, usually checked out +beside this one: `t5de` (patches), `t5de-proxy` (local proxy), later `t5de-catalog` and `t5de-api`. + +## Rules + +- Never commit IMVU's code or captured traffic with account data. Our own declarations of IMVU's protocols and notes + in our own words are fine. +- Test with a throwaway IMVU account, never a real one. +- Commits: author "dhkatz "; never the user's real name or email. +- The user prioritizes backend performance (Go), and wants moderation centered on the hub's website. +- IMVU session keys never leave the user's machine; the hub never asks for them. diff --git a/README.md b/README.md new file mode 100644 index 0000000..75323bc --- /dev/null +++ b/README.md @@ -0,0 +1,6 @@ +# t5de-hub + +T5DE's hub: accounts linked to IMVU accounts, the registry of community products and catalogs, presence, T5DE-hosted +rooms, and the website with the moderation console. Part of T5DE (git.t5de.dev/t5de). + +Design: [docs/design.md](docs/design.md) (draft). No code yet. diff --git a/docs/design.md b/docs/design.md new file mode 100644 index 0000000..671d18a --- /dev/null +++ b/docs/design.md @@ -0,0 +1,284 @@ +# T5DE community design + +Status: draft for review (2026-10-29). Decisions marked **Decided** were agreed in discussion; the rest are +proposals. Open questions are at the end. + +## Goals + +- **Community products**: original work that IMVU's shop doesn't allow, hosted by community-run catalogs, worn and + placed in T5DE without IMVU ever seeing it. +- **One account per person** on a central hub, linked to any number of IMVU accounts. +- **T5DE-hosted rooms**: decorated scenes and their chat, hosted by the hub. +- **Social extras**: an opt-in T5DE badge, opt-in T5DE presence, creator pages. +- **Moderation from one place**: the hub's website. +- **Performance** on every backend. +- **Free first**, with payments possible later without reworking the data. + +Non-goals: re-hosting IMVU's products (original work only), changing what IMVU or non-T5DE users see (they see +IMVU's defaults), and replacing IMVU's servers for anything else (for now). + +## Overview + +``` +┌──────────── user's PC ─────────────┐ +│ T5DE client ── patch ── proxy ─────┼──► IMVU (unchanged; never sees community items) +└────────────────────────────┬───┬───┘ + │ └──► catalogs (many, community-run) ──► S3/R2 storage + ▼ + ┌──────────── hub (T5DE-run) ─────────────┐ + │ API · realtime gateway · rooms · jobs │──► IMVU public API (linking, age checks) + │ website & console · Postgres · NATS · R2│ + └─────────────────────────────────────────┘ +``` + +Repositories (**Decided**): + +| Repository | What | +| --- | --- | +| `t5de` | Client patches; gains removing community items from what IMVU sees, and the new settings. | +| `t5de-proxy` | The local proxy; gains the product resolver, IMQ router, hub and catalog clients. | +| `t5de-hub` | The hub: API, realtime gateway, rooms, jobs, website and moderation console (one Go program). | +| `t5de-catalog` | The reference catalog: API and a small admin page. Anyone can run one. | +| `t5de-api` | Shared definitions: OpenAPI specs, WebSocket messages, token formats, and Go types generated from them. | + +## What the client already does (found 2026-10-29) + +From an experiment with the proxy presenting a made-up product ID as an alias of a real product (t5de-proxy's +local `spike-alias` branch): + +- **Products load from whatever the answers say.** Before using a product the client calls `authorizeProducts2` + (XML-RPC: may this user use it, where are its files, their checksum) and product-info endpoints, then downloads + the files from the addresses given. When the proxy answered for an ID IMVU doesn't know, the client loaded and + rendered the product. +- **IMVU signs requests.** `X-imvu-auth` is md5(user ID + session security key + body), with the query string + added for some calls. A changed request is rejected ("invalid authorization data"). The proxy sees the key in the + login response and could re-sign, but the design avoids sending IMVU anything changed on community products' + behalf: the proxy answers those requests itself, and splits requests that mix both. +- **Outfit changes are chat commands.** Wearing or removing sends plain-text `*use ` / `*remove ` + messages over IMQ (the home screen is a chat session too), and IMVU relays them to everyone in the session. +- **IMVU didn't keep an unknown ID** in the account's saved outfit after it was worn. +- **The proxy holds the IMVU session key.** It never leaves the user's machine: not to the hub, not to catalogs. + +IMVU's public user API (`api.imvu.com/user/user-`) has `is_ageverified`, `is_ap`, `is_adult` (self-reported +age, not verified) and `tagline`. + +## Client side + +### Patch (`t5de`) + +- Removes community product IDs from outgoing `*use` / `*remove` and outfit saves, so IMVU and non-T5DE users see + defaults. Tells the proxy the full outfit through a local endpoint only the proxy answers. +- Settings: T5DE badge, T5DE presence, adult content (each opt-in), hub sign-in. + +### Proxy (`t5de-proxy`) + +- **IMVU passthrough**: as today. +- **Product resolver**: community products are named (global product ID, version) everywhere outside the machine. + The client only understands 32-bit integers (XML-RPC), so the proxy maps each to a local ID in a reserved range + near the top (about 1.9-2.1 billion), stored per install. It answers the client's permission, product-info and + file requests for them, and splits mixed requests (the IMVU part forwarded unchanged). +- **File cache**: files named by sha256, kept on disk; worn items keep working when a catalog is down. +- **IMQ router**: chat for T5DE-hosted rooms goes to the hub, the rest to IMVU. Adds other T5DE users' community + items to incoming `*use` messages, so the client's own code loads them. +- **Hub client**: one WebSocket for presence, room channels and hosted rooms; device-code sign-in, refresh token + encrypted for the Windows user (DPAPI). +- **Local pages**: browsing and account pages shown in the client's built-in browser. +- **Creator live preview**: loads a product from a local folder and reloads it as files change. + +## Identity (**Decided**) + +- A hub **account is a person**. It links to any number of **IMVU accounts**, up to a cap stored per account + (default 10), which moderators can raise on request. +- **Linking**: the hub gives a one-time code, the user puts it in their IMVU tagline, the hub reads it from IMVU's + public API. The user can remove it afterwards. **The most recent proof wins**: linking an account already linked + elsewhere moves it (accounts get sold or given away), and the old link ends. +- **Sign-in**: device-code flow (approve in a browser). Tokens are short-lived Ed25519 JWTs with key IDs; keys + rotate and are published as a JWKS. +- **Alts stay private**: tokens for catalogs and rooms carry an ID per linked IMVU account (pairwise), never the + person's ID, so other people and catalogs can't connect a person's accounts. Only the person and moderators can. +- **Creators** have a public creator profile, separate from (and not revealing) their IMVU accounts. +- **Switching accounts in the client**: the proxy sees which IMVU account logs in and tells the hub. Not linked: + offer to link. Linked to another hub account: no community features that session. +- **Bans** apply to the person, so all linked accounts. + +## Age (**Decided**: IMVU's verification) + +- Adult content requires the **IMVU account in use** to be age verified (`is_ageverified`), checked by the hub + itself (never reported by the proxy, which anyone can change): at linking and at most daily after, cached. + Revocations show up at the next check. +- Per IMVU account, not per person, so one verified account doesn't unlock others a minor might use. +- Enforced at three points: catalogs (adult files need an adult token), the hub (adult items left out of room + channels for viewers who aren't verified and opted in), and the viewer's proxy. +- Self-reported `is_adult` doesn't count. Whether `is_ap` counts is open. + +## Products and catalogs + +- **The hub is the registry.** Products have **global IDs** owned by their creator. A **version** is a file list + (path, sha256, size), signed by the hub on the creator's behalf (creator-held keys later). Versions never change. +- **Catalogs host and curate**: their own rules, ratings ceiling and community. A catalog stores files and tells the + hub which versions it hosts. The same version on several catalogs is mirrored, not duplicated: uploading a file + list identical to an existing version links to it. Variants link with **derived from** (credit, and the original + creator's derive permission). +- **Outfits and rooms refer to (product, version)**, so any catalog hosting it can serve it. +- **Files** are named by hash, immutable, served from object storage or a CDN; never through Go. +- **Ratings**: general and adult, as IMVU. Catalogs declare the highest rating they host. +- **Uploads** go from the creator's browser (or the proxy) straight to the catalog's API with a hub token, so large + files never pass through the hub. The catalog validates them: formats parsed, size limits, nothing executable, + thumbnails generated, extra texture maps accepted (see Better looks). +- **Format**: IMVU's own product files, as the client loads them, so creators keep using IMVU's tools. + +### Catalogs (`t5de-catalog`) + +- Go program, SQLite or Postgres, S3-compatible storage, docker-compose for operators. +- **API**: catalog and search, versions, files by hash, uploads, reports, takedowns. +- **No user-facing site** (**Decided**): users, creators and operators use the hub's website. Operators' rights + come from hub tokens ("operator of catalog X"); the catalog enforces them itself, so operators stay in charge of + their server. +- **Small admin page** for what must work without the hub: first setup, storage, keys, registering with the hub, + health, emergency takedown. +- General-audience files are public (cacheable, and catalogs learn less about who wears what); adult files need a + token. +- Operators can build their own branded site on the public API. + +## Outfits + +1. The user puts on a community item. The client shows it (proxy answers for it); the patch sends IMVU the outfit + without it. +2. The proxy sends the full outfit to the hub, which keeps the current outfit per linked IMVU account and relays + it on the room channel. +3. Other T5DE users' proxies add the items to that user's incoming `*use`, filtered by their own age settings. + Non-T5DE users see the default. + +Saved outfits are per IMVU account (avatars differ). The library (products added, favorites, followed creators) +is per person, shared across their accounts. + +## Rooms (**Decided**: both) + +- **Decorations over IMVU rooms**: community furniture placed in an IMVU room, seen only by T5DE users. Placements + travel on the room channel like outfits. +- **T5DE-hosted rooms**: the room lives on the hub: scene, furniture placement, owner, moderators, permissions, + bans, chat, seats, presence. The proxy routes that room's IMQ traffic to the hub, which speaks the subset of + IMQ the client needs (subscribe, state changes, mounts, messages) and the room-joining XML-RPC calls, so the + client treats it as an ordinary room. Needs our own definitions of those messages, written from observation. +- **Chat retention**: not stored by default; kept briefly only when someone reports it. +- Room shells and furniture are ordinary products. + +## Presence and badge (**Decided**: both opt-in, per IMVU account) + +- **Badge**: other T5DE users see that this account uses T5DE. +- **Presence**: friends see "online in T5DE". The proxy asks the hub about the user's IMVU friends; the hub doesn't + store friend lists, but it sees them when asked (stated in the privacy text). + +## Better looks + +Versions can carry extra texture maps (normal, roughness, ...) beside the standard ones. The proxy hands them to +T5DE's renderer, which uses them when drawing the item; IMVU products and other clients get the standard textures. +Hub and catalogs only store and label the files; the work is in t5de-renderer. + +## Creator pages + +Profile, products across all catalogs, follows, and release notifications (over the realtime gateway). + +## Moderation and policy + +- **One policy at the hub**, binding every catalog to stay listed: original work only; no logos, characters or + real people's likenesses; zero tolerance for sexualized depictions of minors, with the legal reporting duties + that come with it; a takedown process and contact. +- **Levers**: delist a catalog; block a product (global ID, and its file hashes to catch re-uploads) network-wide, + pushed to every proxy; ban a person; close a hosted room; mute or ban in rooms. +- **Console** on the hub's website: report queue, live views (rooms, reports) over server-sent events, actions, + audit log. Catalog operators get their catalog's queue there too. + +## Payments later (**Decided**: free first) + +Built in from the start so payments can be added without reworking data: getting a product creates an +**ownership record** (source "free", price 0); products have a price (always 0); creators have an empty payout +field. A payment service plugs in later. + +## Hub (`t5de-hub`) + +One Go program in modules (split into services only if load requires it): + +- **Identity**: accounts, linked IMVU accounts, link caps, device-code sign-in, tokens, JWKS. +- **IMVU checks**: background jobs reading IMVU's public API (tagline when linking; `is_ageverified` when linking + and at most daily). Called rarely and cached: IMVU could block the hub's address, and nothing but linking may + depend on IMVU being up. +- **Registry**: products, versions, ratings, derivations, catalogs and what they host, search (Postgres full-text + first; Meilisearch when needed). +- **Realtime gateway** (WebSocket): presence, badge, room channels, hosted rooms. Live room state in memory on the + node owning the room; NATS passes it between nodes once there's more than one. +- **Rooms**: hosted rooms' stored state. +- **Moderation**: reports, actions, blocklist, audit log. +- **Website and console** (**Decided**: in this repo): browse catalogs, products, creators, rooms, news; manage + accounts (links, privacy, data export and deletion); catalog operators' screens; moderation console. + Proposed: server-rendered HTML from Go (templ) with htmx, built into the program: fast, cacheable at Cloudflare, + indexable, no JS build. + +### Data (sketch) + +``` +person(id, link_cap, settings, banned_at) +imvu_link(person, imvu_id, pairwise_id, linked_at, age_verified, age_checked_at, badge, presence) +creator(id, person, name, page) +catalog(id, url, operators, max_rating, status, signing_key) +product(id, creator, rating, slot, derived_from, price) +version(product, number, manifest_sha256, signature, extra_maps) +hosting(catalog, version) +ownership(person, product, source, price) +outfit(imvu_link, items[(product, version)]) +room(id, owner, scene, furniture[], roles, bans) +report(...), moderation_action(...), audit(...) +``` + +## Infrastructure + +- **Hub**: two small VMs or containers behind Cloudflare (already in front of Gitea; supports WebSockets), + managed Postgres with point-in-time backups, R2 for the hub's own assets and T5DE's example catalog (no egress + fees). Jobs in Postgres (River). NATS when the gateway has more than one node. +- **Deploys**: Gitea CI builds container images and runs migrations; OpenTofu once there are more than a couple + of machines. +- **Monitoring**: Bugsink (already used for crash reports) for server errors; Prometheus and Grafana (or Grafana + Cloud) for metrics; a status page. +- **Limits**: rate limits at Cloudflare and per account; upload limits at catalogs. + +## Performance + +Go throughout. Product files never pass through Go (hash-named, from a CDN or object storage). pgx with sqlc (plain +SQL, no ORM) and indexes designed up front. Public pages cached at Cloudflare. Live rooms in memory on their node. +The proxy streams and adds no waiting. + +## Failure + +- **Hub down**: community items fall back to IMVU's defaults; everything IMVU keeps working; hosted rooms are + unavailable. +- **Catalog down**: its items come from the proxy's cache or another catalog hosting the same version, else + defaults. +- **IMVU's API down**: linking and age checks wait; existing links and cached checks keep working. + +## Privacy + +- IMVU session keys never leave the user's machine. +- Linked accounts are private (pairwise IDs). +- The hub sees room presence and, for presence, the friends asked about; it stores neither beyond what's needed. +- Catalogs see addresses of those fetching adult files (with a token) and of uploaders. +- All stated in the setup and on the website. + +## Phases + +1. Spec (`t5de-api`); local product packs, removing community items from what IMVU sees, creator live preview. No + servers. +2. Hub identity: linking, age checks, link caps, badge, presence; website basics. +3. Registry and catalogs: creator pages, community outfits visible to T5DE users, moderation console. +4. Decorations over IMVU rooms. +5. T5DE-hosted rooms. +6. Better looks (renderer work; can run alongside 3 or later). +7. Later: payments. + +## Open questions + +- Does `is_ap` (Access Pass) count as age verified? +- Website: templ and htmx, as proposed? +- Catalog listing: open registration with review, or invitation at first? +- The takedown contact and legal entity for the hub. +- Hosted rooms: how much of IMQ the client needs for a room (to be worked out from observation). +- How long reported chat is kept.