diff --git a/docs/console.md b/docs/console.md new file mode 100644 index 0000000..011d249 --- /dev/null +++ b/docs/console.md @@ -0,0 +1,213 @@ +# Permissions, the console, audit and stats + +How people manage what they control on the hub: products, catalogs, importers, hosted rooms, and the hub itself. +The products contract is `products.md`, imports are `imports.md`; this is who may do what, and what they see. + +## Permissions (**Decided**, 2026-11-04) + +- **Code checks permissions, never roles.** Every check is `can(person, permission, target)`. Permissions are + `resource:action` strings (list below). +- **A role is a named bundle of permissions**, given to a person at a **scope**: the hub (everything), a catalog, a + product or a hosted room. +- **Broader scopes cover narrower ones.** A permission held hub-wide covers every catalog, product and room. One held + on a catalog covers that catalog's members, importer, grants and hosting (not the products themselves: delisting + a product from the catalog is `hosting:delist` on the catalog; removing it everywhere is `product:remove`, which + only hub-wide roles have). +- **Owners are derived**, not stored as assignments: a product's creator (`product.creator_id`), a catalog's owner + (`catalog.owner_id`), a hosted room's owner. They can't fall out of step with those columns; they change only by + transfer. +- **Giving roles**: you may give (or take away) a role at a scope only if you hold every permission in it at that + scope, so nobody hands out more than they have. Built-in roles can't be deleted or changed (except by admins, + hub-wide). Every change is audited. +- **Custom roles** (later): the model allows them (a catalog owner making "Member approver" with only + `member:approve`); built-in roles come first. + +``` +role(id, name, scope_type, permissions[], builtin) +role_assignment(person, role, scope_type, scope_id) -- scope_id null for the hub +``` + +### Built-in roles + +| Role | Scope | Permissions | +| --- | --- | --- | +| Admin | hub | all | +| Moderator | hub | `product:*` (not `transfer`), `revision:retract`, `catalog:suspend`, `catalog:delete-review`, `catalog:approve`, `hosting:delist`, `import:grant-on-behalf`, `import:withdraw`, `import:pause`, `case:*`, `person:approve`, `person:ban`, `person:link-cap`, `person:notes`, `audit:read`, `stats:read`, `room:*` | +| Catalog owner (derived) | catalog | `catalog:edit`, `catalog:rotate-key`, `catalog:delete-request`, `catalog:transfer`, `team:manage`, `member:*`, `hosting:delist`, `import:*` (not `grant-on-behalf`), `case:read`, `case:resolve`, `audit:read`, `stats:read` | +| Catalog operator | catalog | as the owner, without `catalog:delete-request`, `catalog:transfer`, `team:manage` | +| Creator (derived) | product | `product:edit`, `product:publish`, `product:delete`, `product:restore`, `product:transfer`, `revision:upload`, `revision:retract`, `hosting:choose`, `case:read`, `audit:read`, `stats:read` | +| Room owner (derived) | hosted room | `room:*` | +| Room moderator | hosted room | `room:kick`, `room:ban`, `room:mute` | + +Hub-wide `admin` and `moderator` keep living in Better Auth's `user.role` (its admin plugin needs it); the permission +check maps them to the bundles above. A creator with `product:edit` can't change fields a moderator locked +(`product:edit-locked`), and lowering a rating is `product:rate-lower` (moderators only; creators ask for a review). + +### Permissions + +| Resource | Actions | +| --- | --- | +| `product` | `edit`, `edit-locked`, `lock`, `rate-lower`, `publish`, `delete`, `restore`, `remove`, `unremove`, `transfer` | +| `revision` | `upload`, `retract` | +| `catalog` | `approve`, `edit`, `rotate-key`, `suspend`, `delete-request`, `delete-review`, `transfer` | +| `team` | `manage` | +| `member` | `approve`, `invite`, `remove`, `ban` | +| `hosting` | `choose`, `delist` | +| `import` | `grant`, `grant-on-behalf`, `accept`, `run`, `pause`, `withdraw` | +| `case` | `read`, `resolve`, `assign` | +| `audit` | `read`, `read-sensitive` | +| `stats` | `read` | +| `person` | `approve`, `ban`, `link-cap`, `notes`, `role`, `view-as` | +| `room` | `edit`, `close`, `kick`, `ban`, `mute` | +| `settings` | `edit` | + +`audit:read-sensitive` (IP addresses, sessions), `person:role`, `person:view-as` and `settings:edit` are admins'. + +### "Why can't I?" + +A refused action answers 403 `forbidden` with `missing: {permission, scope}` (e.g. `{permission: "member:approve", +scope: {catalog: "c1"}}`), and the website says which permission it needed. **View as** (admins, +`person:view-as`): the console as another person sees it, read-only, audited, with a banner; nothing can be changed +while viewing. + +### Two-factor sign-in + +Anyone holding a sensitive permission must have two-factor sign-in on (Better Auth's `twoFactor` plugin, TOTP with +backup codes): `product:remove`, `person:ban`, `person:role`, `audit:read-sensitive`, `catalog:transfer`, +`catalog:suspend`, `import:grant-on-behalf`, `settings:edit`. Without it, those actions answer 403 +`two_factor_required` and the console asks them to set it up. Giving someone such a role reminds them. + +## The console + +One console at `/console` for everyone who controls something: a creator with a product, a catalog's team, a +hosted room's owner or moderators, moderators and admins. Others don't see the link. Every page and action asks +for a permission, and the API checks it again (the website only hides what can't be used). + +Sections (each shown only to those with something in it): + +- **Overview**: charts and to-dos (open cases, member requests, failed imports, drafts) for what the person + controls. +- **Products**: their products (everyone's for moderators), drafts, info history, locks, stats. +- **Catalogs**: the catalogs they're on the team of (all for admins and moderators): settings, key, team, members + and invites, hosted products, cases, the importer (grants, queue, requests) when it has one, health and storage. +- **Rooms**: their hosted rooms. +- **Cases**: reports, appeals and reviews about what they control. +- **People**: moderators and admins. +- **Audit**: scoped (below). +- **Settings**: admins (registration, announcements, roles). + +No catalog is special: importer pages are part of any catalog with imports on; what only admins do (granting on a +creator's behalf) is a hub-wide permission usable on every such catalog. + +## Audit log + +- Every change is an entry: `{at, actor, action, tags, detail}`. `tags` are the resources it touches, e.g. + `{catalog: "c1", product: "p9"}` for "operator delisted p9 from c1", `{person: "x"}` for a ban. +- **Who sees an entry**: anyone with `audit:read` at a scope covering any of its tags. Admins and moderators see + everything. +- **Fields by permission**: IP addresses and sessions only with `audit:read-sensitive`. People appear by hub + username; linked IMVU accounts appear only to moderators (alts stay private). Catalog owners see moderators' + actions on their catalog and their products, with the moderator's name and reason. +- **Filters**: resource, actor, action, date range, text; admins also get "everything a person did" and + "everything that happened to X". Every detail page links to its entries. +- Retention (later): entries kept indefinitely for now; a rule per action type later. + +``` +audit(id, at, actor, action, tags jsonb, detail jsonb, ip) -- GIN index on tags +``` + +The existing `target` column becomes tags (`target` kept for old entries). + +## Cases: reports, appeals and reviews + +One queue for everything someone has to decide. A case has a kind, the resources it's about (tags, as the audit +log), who opened it, a status (`open`, `resolved`, `declined`), an assignee, a resolution and a thread of notes. + +| Kind | Opened by | Decided by | +| --- | --- | --- | +| `report` | anyone signed in, about a product, catalog, room or person | the catalog's team (for products it hosts) and moderators | +| `rating-review` | a creator lowering adult to general | moderators (`product:rate-lower`) | +| `removal-request` | a catalog's team asking to remove a product everywhere | moderators | +| `catalog-deletion` | a catalog's owner (deleting goes to review for now) | moderators (`catalog:delete-review`) | +| `appeal` | the person affected by a removal, ban, delisting or decline (one per action) | a different moderator than the one who acted | + +- People see cases they opened and cases about what they control (`case:read` at a covering scope). +- Reports are limited per person (10 an hour), one open report per person and resource. +- **Canned reasons** (admins edit the list) for removals, bans, declines and edits; free text can be added. +- **Moderator notes**: private notes on people and products (`person:notes`), shown in the console only. +- **Second approval** for the most destructive actions: banning a person, removing a catalog and transferring a + catalog need a second moderator's approval (a case the first opens); admins can act alone in an emergency, which + is flagged in the audit log. + +## Notifications + +One table; shown on the website (a bell, unread count), on the proxy's realtime connection +(`{"type": "notification"}`, so the client can show them later), and by email for types the person opts into. + +Types: a moderator edited your product (with the reason); a case about you or yours was opened or decided; your +membership request was decided; your import finished or failed; your catalog's deletion request was decided; a +product in your library has a new revision; a grant was made or withdrawn; someone requested your products +(daily digest). + +``` +notification(id, person, type, tags, data, created_at, read_at) +notification_setting(person, type, email boolean) +``` + +## API tokens + +Personal tokens for scripts: a name, an expiry (at most a year), and a subset of the person's own permissions at +chosen scopes (e.g. `revision:upload` on two products, `member:approve` on a catalog). Sent as `Authorization: +Bearer t5de_pat_`; only the hash is stored; shown once. A token never has more than its owner holds now: the +check is the intersection, so losing a role takes it from tokens too. Listed and revocable in the account page; +every use updates `last_used_at`; actions are audited as the person, marked "by token ". + +``` +api_token(id, person, name, hash, grants jsonb, expires_at, last_used_at, created_at, revoked_at) +``` + +## Stats and charts + +Charts in the console (shadcn/ui charts, Recharts), following the website's look: neutral, light and dark, color only +for meaning. + +| Who | Charts | +| --- | --- | +| everyone | activity on their resources over time, open cases | +| creators | library adds per product per day, people wearing them this week, import requests | +| catalog teams | members over time, uploads per day, products by rating, storage, cases opened and resolved, importer runs and failures | +| admins | sign-ups and approvals, proxies online, products and revisions per day, library adds, cases, catalogs by status, age checks against the provider's monthly budget | + +- A nightly job rolls counts up into `stat_daily(scope_type, scope_id, metric, day, value)`; "now" figures are live + counters. Charts never scan the raw tables. +- **Only totals.** Counts of people (wearers, adds, members) under 5 show as "fewer than 5", so creators can't pick + out individuals. +- Catalogs may report daily download totals (`POST /api/v1/catalog/stats`); optional, so other people's catalogs + don't have to. +- Client usage stays in Aptabase; these are the hub's own data. + +## Routes + +`/api/v1/console/*` (signed in; each checks its permission). Lists take `limit` and a cursor. + +| | | +| --- | --- | +| `GET /api/v1/console` | `{sections, counts, todo}`: what this person may see | +| `GET /api/v1/console/permissions?scope=...` | the person's permissions at a scope (the website uses it to show actions) | +| `GET /api/v1/console/audit?tag=&actor=&action=&from=&to=&q=` | entries the person may see, newest first | +| `GET /api/v1/console/stats?scope=&metric=&from=&to=` | `{series: [{day, value}]}` | +| `GET/POST /api/v1/cases`, `GET/PATCH /api/v1/cases/:id`, `POST /api/v1/cases/:id/notes` | cases | +| `GET /api/v1/notifications`, `POST /api/v1/notifications/read`, `GET/PUT /api/v1/notifications/settings` | notifications | +| `GET/POST /api/v1/account/tokens`, `DELETE /api/v1/account/tokens/:id` | API tokens | +| `GET/PUT/DELETE /api/v1/roles/:scopeType/:scopeId/:person` | role assignments at a scope | +| `POST /api/v1/console/view-as/:person` | (admins) start viewing as someone, read-only | + +## Later + +- Custom roles and their editor. +- Time-limited role assignments (a temporary moderator for an event). +- Announcement banner and maintenance mode (admins). +- Retention rules for the audit log and soft-deleted records. +- Webhooks (e.g. to Discord) for a catalog's cases and uploads. +- Health alerts for catalog teams when the hub can't reach their catalog; storage quotas. +- CSV export of stats. diff --git a/docs/design.md b/docs/design.md index 9760ce4..d03eb0d 100644 --- a/docs/design.md +++ b/docs/design.md @@ -250,6 +250,16 @@ number, `productdata///`) but hardly shows them; T5DE shows them o the link. - **Seeing versus wearing** (**Decided**): anyone can *see* any catalog's products on someone else (subject to their own age settings); getting and wearing them needs membership where the catalog requires it. +- **Managing catalogs** (**Decided**, 2026-11-04): teams (owner and operators), members and invites, settings, key + rotation, delisting what they host, suspension by moderators; deleting a catalog goes to moderators' review for + now. Revisions can be mirrored to more catalogs. Routes in products.md, permissions in console.md. + +### Imported IMVU products (**Decided**, 2026-11-04) + +Creators who sell on IMVU can grant a catalog permission to import their IMVU products (the whole shop or chosen +ones); the catalog's importer (`t5de-importer`) fetches them from IMVU and hosts them as ordinary T5 products, so +responsibility stays with catalogs and nothing changes for proxies. Anyone can request a product or shop; admins can +record a creator's consent on their behalf. See imports.md. ## In the client's shop and inventory (planned, the user's goal) @@ -346,6 +356,12 @@ Profile, products across all catalogs, follows, and release notifications (over pushed to every proxy; ban a person; close a hosted room; mute or ban in rooms. - **Console** on the hub's website: report queue, live views (rooms, reports) over server-sent events, actions, audit log. Catalog operators get their catalog's queue there too. +- **Permissions** (**Decided**, 2026-11-04): code checks `resource:action` permissions; roles are bundles of them + given at a scope (hub, catalog, product, room). Everyone who controls something gets the console, showing only + what their permissions cover, with a scoped audit log, cases (reports, appeals, reviews), notifications, API + tokens and charts. See console.md. +- **Product info**: new uploads are drafts the creator reviews (title, description, rating, tags, category, listed) + and publishes; moderators can edit any product's info with a reason shown to the creator, and lock fields. ## Payments later (**Decided**: free first) @@ -448,6 +464,23 @@ The proxy streams and adds no waiting. 6. Better looks (renderer work; can run alongside 3 or later). 7. Later: payments. +## Next (2026-11-04) + +Phases for products, catalogs and imports (products.md, console.md, imports.md): + +1. Permissions and console foundations: the permission check, roles, the console for everyone who controls + something, a tagged audit log, membership enforced, catalog settings, team and key rotation, suspension. +2. Product info: drafts, moderator edits with reasons and locks, info history, unlisted products, tags and category. +3. Cases (reports, appeals, reviews), file blocks and the catalogs' changes feed; notifications; API tokens; + two-factor sign-in for sensitive permissions. +4. Imports: grants, recorded grants, requests, unclaimed creators, the job queue, `t5de-importer` (after its spike). +5. Charts and stats; mirroring between catalogs. + +Later: custom roles; time-limited roles; announcements and maintenance mode; audit retention; webhooks; catalog +health alerts and storage quotas; CSV exports; scheduled publishing; revision diffs; product collections; following +creators; favorites and wishlists; search by catalog and gender; outfit links; trying a product on in the client; +previews drawn by t5de-renderer; import dry runs, flattened imports, and the client's "Get on T5DE" button. + ## Open questions - The takedown contact and legal entity for the hub itself. diff --git a/docs/imports.md b/docs/imports.md new file mode 100644 index 0000000..973e1ce --- /dev/null +++ b/docs/imports.md @@ -0,0 +1,167 @@ +# Imported IMVU products + +Creators who sell on IMVU can let T5DE users wear their IMVU products for free: a catalog imports the products from +IMVU, hosts them as ordinary T5 products, and serves them to its members. Decided 2026-11-04 (the user's +discussion). The contract with catalogs and proxies is in `products.md`; permissions and the console are in +`console.md`. + +## Why the catalog imports + +- **Responsibility stays with catalogs**, as for uploads: the catalog that imports a product stores and serves its + files, and its takedown contact answers for them. End users only download from catalogs, as for any T5 product. +- **Nothing changes for proxies**: an imported product is a hub product with revisions. No aliasing of IMVU IDs, no + re-signed IMVU requests. +- **Only consenting creators**: nothing is imported without a grant from the creator (or one an admin records on + their behalf, with evidence). + +## Words + +- **Grant**: a creator's consent for one catalog to import their products: the whole shop, or listed products. +- **Importer** (`t5de-importer`): a program that fetches products from IMVU and uploads them to its catalog. Run by + the catalog's operator beside an ordinary `t5de-catalog`. T5DE runs one for its own import catalog; any catalog + may enable imports. +- **Import job**: one product to import or re-check, queued at the hub. +- **Request**: someone asking for a creator's product, or whole shop, to be on T5DE. +- **Unclaimed creator**: a placeholder creator for IMVU creators without a hub account, whose consent an admin + recorded. Claimed when they join and link that IMVU account. + +## What's imported + +- **The product's own files**: its CFL's entries, with the 16x16 placeholder textures replaced by the full-size + originals its `_contents.json` lists (fetched from IMVU's CDN beside the CFL), and without `_contents.json` (packs + don't use it). Its `index.xml` keeps its `__DATAIMPORT` of the parent. +- **The parent stays on IMVU** (**Decided**, for now): the revision's `parent` is the IMVU product it derives from, + as for uploaded products, and the client loads it from IMVU as it does for anyone's outfit. Option B + (**Decided**): any product the creator made may be imported, whatever it derives from; deriving from a derivable + product is what IMVU's derive permission allows. If keeping the parent on IMVU becomes a problem, imports can be + flattened later (the whole chain in one product, parent 0). +- **Info**: name and rating from IMVU's public API (`api.imvu.com/product/product-`: `product_name`, `rating`, + `creator_cid`), the higher of IMVU's rating and the files' `ProductRating`; a description saying it's imported + with the creator's permission, linking its IMVU page. Creators edit it afterwards like any product. +- **Revisions**: each IMVU revision imported becomes a hub revision ("Imported from IMVU revision 12"). Revision + history and staying on a revision work as for any product. +- One hub product per IMVU product (`product.imvu_product`, unique). Two catalogs importing the same product mirror + the same revisions. + +## Grants + +``` +import_grant(id, catalog, creator person, imvu_id, scope ("shop" | "products"), products bigint[], exclude bigint[], + listed, consent_version, consented_at, consent_kind ("verified" | "recorded"), evidence, recorded_by, + proof_checked_at, accepted_at, accepted_by, withdrawn_at, status ("pending" | "active" | "suspended" | "withdrawn")) +``` + +- **Made by the creator**: on the hub website, "Bring my IMVU shop": choose one of their linked IMVU accounts, a + catalog with imports on, the scope (whole shop with exclusions, or listed products), listed or unlisted, and agree + to the consent text. The text is versioned; the version and time are stored. +- **Accepted by the catalog**: the catalog's team (`import:accept`) accepts or declines it, so nobody pushes imports + onto a catalog without its operator agreeing. Until then it's `pending`. +- **Recorded by an admin** (`import:grant-on-behalf`, hub-wide; for creators who won't make an account): the IMVU + creator (ID or name), scope, catalog, and **evidence** (required: how they agreed, e.g. "Discord DM, 2026-11-04", + optionally a screenshot). Optional **proof without an account**: the console gives a code to send the creator; + they put it in their IMVU tagline, and the hub checks it as for linking. Shown as "consent verified" or "consent + recorded by admin". The products belong to an unclaimed creator. +- **Scope "shop"**: any product whose `creator_cid` is the grant's IMVU account, including new ones, except + exclusions. **Scope "products"**: only those listed; the creator can add more (requests show as suggestions). +- **Changes**: the creator (or an admin, for recorded grants) changes the scope, list and exclusions any time. + Removing a product from a grant withdraws it (below). +- **Ending**: the creator withdraws the grant, or the catalog's team ends it, or the IMVU account's link moves to + another person (the grant is `suspended` until the new owner consents again), or the account is unlinked + (`suspended`). Admins can withdraw any grant. + +### Unclaimed creators and claiming + +An unclaimed creator is a hub person without credentials (can't sign in), named after the IMVU account ("Bar, on +IMVU"), marked unclaimed. When someone links that IMVU account, the hub offers them the claim: the products, grants, +requests and stats move to them, and the placeholder is deleted. Until then, withdrawal goes through admins +(contacting T5DE, or the catalog's takedown contact). + +## Requests + +``` +import_request(person, imvu_creator, imvu_product (0: the whole shop), created_at) -- unique per person and target +``` + +- Anyone signed in can request a product (by IMVU ID or from its shop page in the client) or a creator's whole shop. + At most 50 a day per person; one per person and target, so counts can't be inflated. +- The hub looks up the product's creator (public API, cached in `imvu_product_info`). +- **Covered by an active grant**: queued for import at once; the person gets the product in their library when it's + done (pending until then; nothing on screen waits for it). +- **Not covered, grant by products**: a suggestion for the creator ("12 people asked for X"), added with one click. +- **No grant**: counted. The creator sees the demand when they bring their shop; the hub's creator page for a shop + that hasn't joined can show "requested by 47 people". The hub never contacts creators through IMVU. +- Admins see request counts per IMVU creator in the console (whom it's worth asking). + +## Import jobs + +``` +import_job(id, catalog, grant, imvu_product, kind ("import" | "recheck"), status ("queued" | "claimed" | "done" | "failed"), + imvu_revision, reason, attempts, claimed_at, next_at, created_at, updated_at) +``` + +- **Queued by the hub**: a request covered by a grant, a product added to a grant, a "shop" grant accepted (the + products people requested so far, and all the shop's products if a way to list them is found), and a daily + **recheck** of each imported product. +- **The hub checks IMVU's public API** before queuing and at each recheck (rate-limited, as for parents): the + product exists, its creator is the grant's account, its rating. Gone from IMVU: withdrawn. Rating changed to AP: + adult at once. Retired (`is_purchasable` false): kept unless the creator excludes it. +- **The importer claims jobs**, fetches, converts, uploads with a `t5de-import` token (products.md), and reports the + outcome. Rechecks probe IMVU's CDN for revisions after the last imported one; a new one is imported. +- Failures retry with backoff (1 min, 10 min, 1 h, then daily; failed for good after 5); the reason is shown in the + console. The catalog's team can retry or re-import one product, and pause the importer (`import:pause`): the hub + hands out no jobs while paused. + +## Withdrawal + +A withdrawn imported product (grant withdrawn or narrowed, gone from IMVU, account link moved) is removed as +moderation removes products: it leaves libraries, outfits and rooms; its page says it was withdrawn; the catalog +deletes its files (the catalog changes feed, products.md). Unlike deleting an uploaded product, people who had it +don't keep it: the consent is gone. + +## `t5de-importer` + +A Go program (its own repository) beside a `t5de-catalog`; it stores nothing lasting but its working files. + +- Configuration: `HUB_URL`, `CATALOG_ID`, `IMPORTER_KEY` (from enabling imports on the catalog), `IMPORTER_RATE` + (requests a second to IMVU, default 2), optionally an IMVU account for exact revisions (`IMVU_USERNAME`, + `IMVU_PASSWORD`; the operator's choice and risk; the hub never holds IMVU credentials). +- Revisions: without an IMVU account, the CDN is probed (`HEAD productdata//`) once per product; afterwards a + recheck tries the next few numbers. With one, `authorizeProducts` gives exact revisions. +- Reads CFLs with t5de-api's `cfl` package; the conversion follows `@imvu/chkn` (dhkatz/imvu.js) as a reference. +- Validates with t5de-api's `pack` rules before uploading; a product breaking them fails with the reason. (The spike + checks whether typical IMVU products fit; imports may need their own limits.) + +## Hub routes + +| | | +| --- | --- | +| `GET /api/v1/imports/catalogs` | approved catalogs with imports on, accepting grants | +| `POST /api/v1/imports/grants` | `{catalog, imvuId, scope, products?, exclude?, listed, consentVersion}` → 201 `{grant}` (pending). With `onBehalf: {imvuId, evidence, proof?}` instead of a linked `imvuId`: an admin-recorded grant. | +| `GET /api/v1/imports/grants` | the person's grants; with `catalog=`, a catalog's (its team) | +| `PATCH /api/v1/imports/grants/:id` | `{scope?, products?, exclude?, listed?}` | +| `POST /api/v1/imports/grants/:id/accept`, `.../decline` | the catalog's team | +| `DELETE /api/v1/imports/grants/:id` | withdraw | +| `POST /api/v1/imports/proofs` | (admins) `{imvuId}` → `{proof, code}`; `POST .../proofs/:id/check` | +| `POST /api/v1/imports/requests` | `{imvuProduct}` or `{imvuCreator}` → `{status: "queued" \| "suggested" \| "counted", product?}` | +| `GET /api/v1/imports/requests?imvuCreator=` | counts (the creator, admins) | +| `GET /api/v1/imports/available?imvuProducts=a,b` | for the proxy's shop button: which can the person get (and the hub product ID), already have, or request | +| `GET /api/v1/imports/jobs?catalog=&status=` | the catalog's team: the queue; `POST .../jobs/:id/retry` | +| `POST /api/v1/imports/claim` | `{imvuId}`: claim an unclaimed creator's products for a linked account | + +For importers (`Authorization: Importer :`) in products.md. + +## Spike (before building the importer) + +1. Import one product by hand as a local pack (full-size textures, no `_contents.json`, parent kept). Does the + client wear it as a T5 product, with the parent loading from IMVU? (Never committed: it's IMVU's files.) +2. Do typical IMVU products fit the `pack` limits? +3. How many HEAD requests does probing take for real shops; does the CDN limit them? +4. Is there a public way to list a creator's products (for importing a whole shop at once)? + +## Later + +- Flattened imports (parent included), if keeping the parent on IMVU causes problems. +- A dry run: preview an import (and anything over the pack limits) before it's published. +- "Get on T5DE" on IMVU's shop and product pages in the client (a t5de patch, using + `/api/v1/imports/available`). +- Creators who don't want their products as parents of imports (an opt-out hiding imports that derive from theirs). diff --git a/docs/products.md b/docs/products.md index 3136c69..2e4f507 100644 --- a/docs/products.md +++ b/docs/products.md @@ -14,6 +14,17 @@ what each side sends. moderator approves it. - **Library**: the products a person "got" (free), per person (all their IMVU accounts), optionally staying on a revision. +- **Draft**: a new product before its creator publishes it: only its creator and moderators see it; it can't be + got, worn or placed. Drafts never published are deleted after 30 days. +- **Unlisted**: a published product left out of search, lists and creator pages for others; anyone with its link + can see and get it (membership rules apply). +- **Member**: a person a catalog with approval or invite membership has let in. A product is **available** to a + person if a catalog hosting it is approved, not suspended, and open or has them as a member (and hasn't banned + them). +- **Imported product**: a product a catalog's importer brought from IMVU with its creator's grant (imports.md). + `imvuProduct` is set; otherwise it's like any product. + +Who may do what (creators, catalog teams, moderators) is in `console.md` ("Permissions"). ## Tokens @@ -26,6 +37,7 @@ unknown `kid`). Claims always include `iss` (the hub's API URL), `typ` (below) a | `t5de-upload` | `sub` (person ID), `catalog` (catalog ID), `product` (optional: a new revision of this product) | 30 min | the creator's browser uploads to the catalog | | `t5de-access` | `sub` (person ID), `adult` (bool) | 1 h | the proxy fetches adult files from catalogs | | `t5de-revision` | `product`, `revision`, `rating`, `files` (the file list), `parent` (IMVU product ID or 0) | none (`exp` absent) | the hub vouches for a revision's files | +| `t5de-import` | `sub` (the job ID), `catalog`, `imvuProduct`, `imvuRevision` | 30 min | an importer uploads an imported product to its catalog | ## Catalog API (`t5de-catalog`) @@ -38,6 +50,14 @@ Served at the catalog's public URL. CORS: the hub website's origin may call `POS | `GET /api/v1/files/` | the file's bytes, `Cache-Control: public, max-age=31536000, immutable` and `ETag` = the hash. A file used only by adult revisions needs `Authorization: Bearer ` with `adult: true` (else 401/403, and `Cache-Control: private`). 404 if the catalog doesn't have it. | | `GET /api/v1/revisions//` | `{product, revision, rating, files}`, for revisions the catalog hosts | +Imports: `POST /api/v1/uploads` also takes `Authorization: Bearer ` (from the catalog's +importer). Then `notes` is optional, and the catalog registers the revision with `importToken` instead of +`uploadToken`. Validation is the same. + +Changes: the catalog polls the hub's changes feed (below) every minute and acts on it: deletes the files of +revisions withdrawn or removed (files still used by another revision it hosts stay), and fetches revisions it's +asked to mirror. + Configuration (environment): `CATALOG_ADDR` (listen address, default `127.0.0.1:4102`), `CATALOG_DATA` (data folder: SQLite database and files), `CATALOG_PUBLIC_URL`, `HUB_URL` (the hub's API), `CATALOG_ID` and `CATALOG_KEY` (from registering on the hub), `WEB_ORIGIN` (the hub website, for CORS). @@ -50,7 +70,21 @@ registering on the hub), `WEB_ORIGIN` (the hub website, for CORS). | | | | --- | --- | -| `POST /api/v1/catalog/revisions` | `{uploadToken, files, rating ("general" or "adult"), parent (IMVU product ID or 0), name (from index.xml, for a new product), notes}` → 201 `{product, revision}`. The upload token must be for this catalog. Without `product` in the token, a new product is made (owner: the token's person, name from `name`). With it, a new revision of that product (the person must own it; `notes` required). A file list identical to the latest revision's is refused (409 `unchanged`). | +| `POST /api/v1/catalog/revisions` | `{uploadToken, files, rating ("general" or "adult"), parent (IMVU product ID or 0), name (from index.xml, for a new product), notes}` → 201 `{product, revision, draft}`. The upload token must be for this catalog. Without `product` in the token, a new product is made as a **draft** (owner: the token's person, name from `name`; the creator publishes it on the website). With it, a new revision of that product (the person must own it; `notes` required). A file list identical to the latest revision's is refused (409 `unchanged`). Files the hub blocks are refused (409 `blocked`). The catalog's upload policy (`anyone`, `members`, `team`) is checked when the upload token is made (403 `forbidden`). With `importToken` instead of `uploadToken`: an imported revision (imports.md); the hub checks the job and that a grant still covers it, makes the product (published, owned by the grant's creator) or adds the revision, and marks the job done. | +| `GET /api/v1/catalog/changes?after=` | `{changes, cursor}`: what the catalog must act on, oldest first: `{type: "delete", product, revision, files}` (withdrawn, removed by moderation, blocked: delete those files unless another hosted revision uses them), `{type: "mirror", product, revision, files, from: [catalog URLs]}` (fetch each file from one of `from`, check its sha256, then confirm below), `{type: "status", status}` (the catalog was suspended or reinstated). Start with no `after`; keep the cursor. | +| `POST /api/v1/catalog/hosting` | `{product, revision}`: the catalog now hosts a revision it was asked to mirror (it has every file) → 204 | +| `POST /api/v1/catalog/stats` | optional: `{day, downloads, bytes}` for a past day | + +### For importers (server to server) + +`Authorization: Importer :` (the key from enabling imports on the catalog; rotated like the +catalog's). Only approved catalogs with imports on and not paused. + +| | | +| --- | --- | +| `POST /api/v1/importer/jobs/claim` | `{limit}` (1-20) → `{jobs: [{id, kind, imvuProduct, lastRevision}]}`; claimed jobs not reported within 30 minutes go back to the queue | +| `POST /api/v1/importer/jobs/:id/token` | `{imvuRevision}` → `{token, url}`: a `t5de-import` token and the catalog's upload URL | +| `POST /api/v1/importer/jobs/:id` | `{status: "done" \| "failed" \| "unchanged", imvuRevision?, reason?}` | ### For people (website, signed in; the proxy with its bearer token) @@ -58,15 +92,36 @@ registering on the hub), `WEB_ORIGIN` (the hub website, for CORS). | --- | --- | | `POST /api/v1/catalogs` | register a catalog: `{name, url, contact, visibility ("public"/"private"), membership ("open"/"approval"/"invite")}` → 201 `{catalog, key}` (the key is shown once) | | `GET /api/v1/catalogs` | (anyone) approved public catalogs; the person's own, any status | -| `POST /api/v1/products/upload` | `{catalog, product?}` → `{token, url}`: an upload token and the catalog's upload URL | -| `GET /api/v1/products?q=&creator=&category=&limit=` | (anyone) `{products}`, newest first; adult ones only for people with an adult account. `category`: an IMVU category ID; only products whose latest revision's parent has it anywhere in its category path (e.g. 106 Clothing, 128 Tops). `limit`: 1-200, default 60 | -| `GET /api/v1/products/:id` | (anyone, adult rule as above) `{product, revisions, inLibrary?, mine?}` | -| `PATCH /api/v1/products/:id` | (creator) `{name?, description?, rating?}`; description is HTML, cleaned by the allowlist; rating may go general → adult | +| `GET /api/v1/catalogs/:id` | (anyone, if public or they're a member or on its team) `{catalog, membership?: {status}, permissions}` | +| `PATCH /api/v1/catalogs/:id` | (`catalog:edit`) `{name?, url?, contact?, visibility?, membership?, maxRating? ("general"/"adult"), uploads? ("anyone"/"members"/"team"), mirrors? ("auto"/"approve"/"off")}`. A new `url` must answer `/health`. | +| `POST /api/v1/catalogs/:id/key` | (`catalog:rotate-key`) → `{key}`: a new key; the old one stops working at once | +| `POST /api/v1/catalogs/:id/importer` | (`import:run`) `{enabled}` → `{key?}`: turns imports on (a new importer key, shown once) or off; `POST .../importer/key` rotates it; `{paused}` pauses | +| `GET /api/v1/catalogs/:id/team`, `PUT/DELETE .../team/:person` | (`team:manage`) `{role: "operator"}` | +| `POST /api/v1/catalogs/:id/transfer` | (`catalog:transfer`) `{to}`: offered to the person, who accepts on their side (`POST .../transfer/accept`) | +| `POST /api/v1/catalogs/:id/delete` | (`catalog:delete-request`) `{reason}`: opens a `catalog-deletion` case; nothing changes until a moderator decides (console.md) | +| `POST /api/v1/catalogs/:id/join` | join an open catalog, or ask to join an approval one; `{invite}` to use an invite code. `DELETE` leaves. | +| `GET /api/v1/catalogs/:id/members?status=` | (`member:approve`) members, requests and bans | +| `PUT /api/v1/catalogs/:id/members/:person` | `{status: "member" \| "banned"}` (`member:approve` / `member:ban`); `DELETE` removes (`member:remove`) | +| `POST /api/v1/catalogs/:id/invites` | (`member:invite`) `{maxUses?, expiresIn?}` → `{invite: {code, url}}`; `GET` lists, `DELETE .../invites/:code` revokes | +| `GET /api/v1/catalogs/:id/products` | the products it hosts (its team also sees drafts' counts, delisted ones) | +| `DELETE /api/v1/catalogs/:id/products/:product` | (`hosting:delist`) `{reason}`: the catalog stops hosting every revision of it; it's in the changes feed. The product stays available from other catalogs hosting it. | +| `POST /api/v1/catalogs/:id/suspend`, `.../reinstate` | (`catalog:suspend`) `{reason}`: a suspended catalog's products aren't available from it, it can't register revisions, and its team sees why | +| `POST /api/v1/products/upload` | `{catalog, product?}` → `{token, url}`: an upload token and the catalog's upload URL (the catalog's upload policy and rating ceiling apply) | +| `GET /api/v1/products?q=&creator=&category=&catalog=&tag=&limit=` | (anyone) `{products}`, newest first: published and listed; adult ones only for people with an adult account. `category`: an IMVU category ID; the product's own category if set, else its latest revision's parent's category path (e.g. 106 Clothing, 128 Tops). `limit`: 1-200, default 60 | +| `GET /api/v1/products/:id` | (anyone, adult rule as above; drafts only for their creator and moderators) `{product, revisions, inLibrary?, mine?, available, permissions}` | +| `PATCH /api/v1/products/:id` | (`product:edit`) `{name?, description?, rating?, tags?, category?, listed?, reason?}`; description is HTML, cleaned by the allowlist; rating may go general → adult; adult → general opens a `rating-review` case unless the person has `product:rate-lower`. `reason` is required when the person isn't the creator, and shown to them. Locked fields need `product:edit-locked`. Each change is kept in the info history. | +| `POST /api/v1/products/:id/publish` | (`product:publish`) a draft becomes published; it needs a name, and a rating no lower than its files declare | +| `GET /api/v1/products/:id/history` | the info history: `{entries: [{at, by, fields, reason}]}`; `POST .../history/:n/revert` | +| `PUT /api/v1/products/:id/locks` | (`product:lock`) `{fields, reason}` | +| `POST /api/v1/products/:id/transfer` | (`product:transfer`) `{to}`; the other person accepts (`.../transfer/accept`) | +| `POST /api/v1/products/:id/restore` | (`product:restore`) undo a creator's delete | +| `POST /api/v1/products/:id/hosting` | (`hosting:choose`) `{catalog, revision?}`: ask another catalog to mirror it (every revision, or one); the catalog's `mirrors` setting decides (auto, or its team approves) | | `POST /api/v1/products/:id/revisions/:n/retract` | (creator) | -| `DELETE /api/v1/products/:id` | (creator) deletes it: no longer listed, and nobody new can get it; people who have it keep it (its page, files and library entries stay, and rooms keep showing it). Its creator can't edit it or add revisions after. 204 | -| `POST /api/v1/products/:id/remove` | (moderator) `{reason}`: removes it everywhere: its page and files answer 410 `removed`, and it leaves libraries and rooms | -| `GET /api/v1/library` | `{library: [{product, revision, stayOn}]}`: the person's products, each with the revision to use | -| `PUT /api/v1/library/:id` | get a product (add it) | +| `DELETE /api/v1/products/:id` | (creator) deletes it: no longer listed, and nobody new can get it; people who have it keep it (its page, files and library entries stay, and rooms keep showing it). Its creator can't edit it or add revisions after (until restored). 204 | +| `POST /api/v1/products/:id/remove` | (`product:remove`) `{reason}`: removes it everywhere: its page and files answer 410 `removed`, it leaves libraries and rooms, and catalogs delete its files. `.../unremove` (`product:unremove`) undoes it while catalogs still have the files. | +| `PUT /api/v1/blocks/:sha256` | (`product:remove`) `{reason}`: blocks a file network-wide: revisions with it can't be registered, and products using it are removed. `DELETE` unblocks. | +| `GET /api/v1/library` | `{library: [{product, revision, stayOn}]}`: the person's products that are available to them, each with the revision to use (products no longer available, e.g. after leaving a catalog, stay in the table but aren't listed until they are again) | +| `PUT /api/v1/library/:id` | get a product (add it): it must be published and available to the person (403 `members_only`, with the catalogs they could join) | | `DELETE /api/v1/library/:id` | remove it | | `PATCH /api/v1/library/:id` | `{stayOn: n or null}` | | `GET /api/v1/products/:id/files` | for the proxy: `{product, revision, rating, parent, name, files, catalogs: [url], token}` for the revision this person uses (their pin, else the latest that isn't retracted); `token` is the `t5de-revision` token | @@ -75,7 +130,8 @@ registering on the hub), `WEB_ORIGIN` (the hub website, for CORS). The realtime connection also tells the proxy `{"type": "library"}` when the person's library changes. A product (in lists, its page, the library) is `{id, name, description, rating, latest, creator, creatorUsername, -parent, image, categories, createdAt, updatedAt}`. `parent`, `image` and `categories` are the latest revision's: +parent, image, categories, status ("draft"/"published"), listed, tags, category, locked, imvuProduct, createdAt, +updatedAt}`. `parent`, `image` and `categories` are the latest revision's: `parent` is the IMVU product it derives from (0 if none); `image` (an https address on `*.imvu.com`, or null) and `categories` (`[{id, name}]`, outermost first, e.g. Clothing / Clothing for Female / Tops / Coats & Jackets; `[]` if unknown) are that IMVU product's, from IMVU's public API (`GET https://api.imvu.com/product/product-`). @@ -99,7 +155,9 @@ every 10 minutes, 20 at a time. adult revisions), checks each sha256, and writes the product as a pack in its cache (`%APPDATA%\T5DE\proxy\hub-products\\`: `t5de-product.json` with the hub's name and rating, and `files/`). From there it's served like any pack. -- Products others wear in a room that it doesn't have are fetched the same way, then given to the client. +- Products others wear in a room that it doesn't have are fetched the same way, then given to the client. Seeing + needs no membership (design.md, "Seeing versus wearing"): `/files` answers for any published product the person + may see by rating. Wearing does: the hub relays a person's outfit with only the products available to them. - Library products appear in the client's inventory (see the proxy's README). ## Decorations