# t5de-catalog

T5DE's catalog server: it stores community products' files by sha256, serves them, and registers uploads with
T5DE's hub (git.t5de.dev/t5de/t5de-hub). Creators upload from the hub's website straight to a catalog, so large
files never pass through the hub; T5DE's proxy then downloads the files from the catalogs the hub lists. Anyone
can run a catalog and ask the hub to approve it.

The contract (tokens, this API, the hub's API for catalogs) is t5de-hub's `docs/products.md`; the design and the
reasons are in its `docs/design.md` ("Products and catalogs", "Catalogs (`t5de-catalog`)").

Unofficial: not made by or affiliated with IMVU.

## How it works

- **Uploads** (`POST /api/v1/uploads`): the creator's browser sends the product as a zip (Create Mode's `.chkn`,
  files at the root) with an upload token from the hub. The catalog checks the token (signed by the hub, for this
  catalog), checks the files by t5de-api's pack rules (names, limits counted while reading, `index.xml` with
  `ProductRating` GA or AP), stores each file under its sha256, and registers the revision with the hub
  (`POST /api/v1/catalog/revisions`, `Authorization: Catalog <id>:<key>`). Only once the hub accepts it is the
  revision recorded as hosted; files of a refused upload stay on disk but aren't served.
- **Files** (`GET /api/v1/files/<sha256>`): immutable, so cached for a year (`ETag` is the hash; ranges work). A
  file used only by adult revisions needs an access token from the hub with `adult: true`, and isn't cached by
  shared caches.
- **Tokens** are EdDSA (Ed25519) JWTs; the hub's keys come from `HUB_URL/.well-known/t5de-keys.json`, cached and
  fetched again when a token names an unknown key (at most every 30 s).
- **Storage**: `CATALOG_DATA/catalog.db` (SQLite: the revisions hosted and which files belong to them) and
  `CATALOG_DATA/files/<first 2 hex>/<sha256>`, written whole (temp file, then rename). `CATALOG_DATA/tmp` holds
  uploads being checked and is emptied at start.

## API

| | |
| --- | --- |
| `GET /health` | `{ok: true}` |
| `POST /api/v1/uploads` | `Authorization: Bearer <t5de-upload token>`; multipart: `file` (the zip), `notes` (required for a new revision of an existing product). 201 `{product, revision}`. Errors are `{error, message}`: 400 bad files or form (`bad_files`, `notes_required`, `bad_request`), 401 bad token (`bad_token`), 413 over 300 MiB (`too_large`), 502 the hub refused (its error code, e.g. `unchanged`) or couldn't be reached (`hub_unavailable`). CORS for `WEB_ORIGIN`. |
| `GET /api/v1/files/<sha256>` | the file; 401/403 for adult-only files without an adult access token; 404 if not hosted |
| `GET /api/v1/revisions/<product>/<revision>` | `{product, revision, rating, files}` for revisions hosted here |

## Running

Register the catalog on the hub's website (`POST /api/v1/catalogs`) to get its ID and key, and have a hub
moderator approve it. Then, for a hub running locally:

    HUB_URL=http://localhost:4101 \
    CATALOG_ID=<id> CATALOG_KEY=<key> \
    CATALOG_PUBLIC_URL=http://localhost:4102 \
    WEB_ORIGIN=http://localhost:4100 \
    CATALOG_DATA=./data \
    go run ./cmd/t5de-catalog

| Variable | |
| --- | --- |
| `CATALOG_ADDR` | listen address (default `127.0.0.1:4102`) |
| `CATALOG_DATA` | data folder (default `./data`) |
| `CATALOG_PUBLIC_URL` | the catalog's public URL, as registered on the hub (logged) |
| `HUB_URL` | the hub's API; also the tokens' issuer (required) |
| `CATALOG_ID`, `CATALOG_KEY` | from registering on the hub (required) |
| `WEB_ORIGIN` | the hub website's origin, allowed to upload from browsers |

It logs to standard error and stops cleanly on Ctrl+C or SIGTERM, finishing requests in progress. Put it behind
a TLS-terminating reverse proxy (or a CDN for `/api/v1/files/`) in production.

With Docker:

    docker build -t t5de-catalog .
    docker run -p 4102:4102 -v catalog-data:/data -e HUB_URL=... -e CATALOG_ID=... -e CATALOG_KEY=... \
      -e CATALOG_PUBLIC_URL=... -e WEB_ORIGIN=... t5de-catalog

## Building

    go test ./...
    CGO_ENABLED=0 go build -trimpath -o dist/t5de-catalog ./cmd/t5de-catalog

The SQLite driver is pure Go (modernc.org/sqlite), so builds are static. t5de-api comes from git.t5de.dev
directly: set `GOPRIVATE=git.t5de.dev`.