feat: file blocks (#10)
Phase 3, part 2 (`docs/design.md`, "Next"): file blocks. Moderators block a file network-wide by its sha256, so a removed product can't come back under another name or catalog.
**Blocking**
- A file's page (Console → Blocks → a file) shows whether it's blocked and the products using it, so a moderator sees what a block removes before confirming ("Block and remove 2 products").
- Moderators reach it from a product's files (each has a Block link for them) or by pasting a sha256 into the Blocks tab, which also lists blocked files.
- Blocking a file:
- removes every product with a revision using it. Each removal is audited as `product.remove` with the file, so creators are notified and can appeal it as before.
- puts a `{type: "block", sha256}` change in the changes feed of every catalog hosting it, meaning: delete it whatever uses it. (t5de-catalog doesn't read the feed yet.)
- refuses uploads with it from then on (409 `blocked`). Each attempt is audited as `file.blocked_upload` with the person and catalog, so moderators see who tried.
- Unblocking lets the file in again. The products it removed stay removed until a moderator restores them, one by one.
- Who: `product:remove` hub-wide (moderators and admins). The Blocks tab shows only for them.
**API**: `GET /api/v1/blocks`, and `GET`/`PUT`/`DELETE /api/v1/blocks/:sha256` (`docs/products.md`).
**Migration** `0013_file_blocks`: the `file_block` table, and a GIN index on revisions' file lists, so finding the products using a file doesn't scan every revision.
**Checks**
- 53 API tests pass. The new `blocks.test.ts` covers:
- who may block
- the preview of what a block removes
- removal of every product using the file, across catalogs and creators
- the changes feed, notifications and appeals
- refused uploads and their audit entry
- unblocking, and restoring a removed product afterwards
- Type checks pass.
- On a local hub, through the website's own forms:
- the Block link on a product's files, for moderators only
- the Blocks tab, for moderators only (404 for the creator)
- the file's page, then blocking: the product is gone, its creator notified and offered Appeal
- a re-upload through the catalog API: refused, and audited
- unblocking, then Restore
- Fixed along the way: an empty reason gave a generic "Some of that isn't right"; it now says "Give a reason."
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Phase 3, part 2 (`docs/design.md`, "Next"): file blocks. Moderators block a file network-wide by its sha256, so a removed product can't come back under another name or catalog.
**Blocking**
- A file's page (Console → Blocks → a file) shows whether it's blocked and the products using it, so a moderator sees what a block removes before confirming ("Block and remove 2 products").
- Moderators reach it from a product's files (each has a Block link for them) or by pasting a sha256 into the Blocks tab, which also lists blocked files.
- Blocking a file:
- removes every product with a revision using it. Each removal is audited as `product.remove` with the file, so creators are notified and can appeal it as before.
- puts a `{type: "block", sha256}` change in the changes feed of every catalog hosting it, meaning: delete it whatever uses it. (t5de-catalog doesn't read the feed yet.)
- refuses uploads with it from then on (409 `blocked`). Each attempt is audited as `file.blocked_upload` with the person and catalog, so moderators see who tried.
- Unblocking lets the file in again. The products it removed stay removed until a moderator restores them, one by one.
- Who: `product:remove` hub-wide (moderators and admins). The Blocks tab shows only for them.
**API**: `GET /api/v1/blocks`, and `GET`/`PUT`/`DELETE /api/v1/blocks/:sha256` (`docs/products.md`).
**Migration** `0013_file_blocks`: the `file_block` table, and a GIN index on revisions' file lists, so finding the products using a file doesn't scan every revision.
**Checks**
- 53 API tests pass. The new `blocks.test.ts` covers:
- who may block
- the preview of what a block removes
- removal of every product using the file, across catalogs and creators
- the changes feed, notifications and appeals
- refused uploads and their audit entry
- unblocking, and restoring a removed product afterwards
- Type checks pass.
- On a local hub, through the website's own forms:
- the Block link on a product's files, for moderators only
- the Blocks tab, for moderators only (404 for the creator)
- the file's page, then blocking: the product is gone, its creator notified and offered Appeal
- a re-upload through the catalog API: refused, and audited
- unblocking, then Restore
- Fixed along the way: an empty reason gave a generic "Some of that isn't right"; it now says "Give a reason."
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: dhkatz <[email protected]>
This commit was merged in pull request #10.
This commit is contained in:
dhkatz authored and 
dhkatz committed
1 parent
7299e10b08
commit
58a3e70a63
18 files changed
+4074
-10
No files matched your search
@@ -0,0 +1,203 @@
|
||||
// File blocks (docs/products.md): moderation blocks a file network-wide; products using it are removed, catalogs told
|
||||
// to delete it, and uploads with it refused.
|
||||
import { afterAll, beforeAll, expect, test } from "bun:test";
|
||||
import { openDatabase, type Database } from "@t5de/db";
|
||||
import { createApp } from "../src/app";
|
||||
import { createAuth } from "../src/auth";
|
||||
import { ImvuClient, type ImvuUser } from "../src/imvu";
|
||||
|
||||
const webURL = "http://web.test";
|
||||
const apiURL = "http://api.test";
|
||||
const accounts = new Map<number, ImvuUser>();
|
||||
const fakeIMVU: typeof fetch = (async (input: RequestInfo | URL) => {
|
||||
const url = new URL(String(input));
|
||||
if (url.pathname.startsWith("/product/")) return Response.json({ status: "failure" }, { status: 404 });
|
||||
const u = [...accounts.values()].find(
|
||||
(u) => url.pathname === `/user/user-${u.id}` || url.searchParams.get("username")?.toLowerCase() === u.username.toLowerCase(),
|
||||
);
|
||||
return Response.json({
|
||||
denormalized: u
|
||||
? { [`https://api.imvu.com/user/user-${u.id}`]: { data: { legacy_cid: u.id, username: u.username, tagline: u.tagline, is_ageverified: u.ageVerified, is_ap: u.accessPass } } }
|
||||
: {},
|
||||
});
|
||||
}) as typeof fetch;
|
||||
|
||||
// Catalog addresses that answer /health (for address changes).
|
||||
const healthy = new Set<string>(["https://moved.example.test"]);
|
||||
|
||||
let db: Database;
|
||||
let close: () => Promise<void>;
|
||||
let base: string;
|
||||
let stop: () => void;
|
||||
|
||||
beforeAll(async () => {
|
||||
({ db, close } = await openDatabase("memory"));
|
||||
const auth = createAuth(db, { apiURL, webURL, secret: "test-secret-test-secret-test-secret-1234", rateLimit: false });
|
||||
const hub = createApp({
|
||||
db,
|
||||
auth,
|
||||
imvu: new ImvuClient("http://imvu.test", fakeIMVU),
|
||||
apiURL,
|
||||
webURL,
|
||||
catalogHealthy: async (url) => healthy.has(url),
|
||||
});
|
||||
hub.app.listen({ port: 0, hostname: "127.0.0.1" });
|
||||
const server = hub.app.server as unknown as { port: number; stop: () => void };
|
||||
stop = () => server.stop();
|
||||
base = `http://127.0.0.1:${server.port}`;
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
stop();
|
||||
await close();
|
||||
});
|
||||
|
||||
type Person = { cookie: string; name: string };
|
||||
async function call(p: Person | undefined, method: string, path: string, body?: unknown, headers: Record<string, string> = {}) {
|
||||
const res = await fetch(base + path, {
|
||||
method,
|
||||
headers: { Origin: webURL, ...(body !== undefined ? { "Content-Type": "application/json" } : {}), ...(p ? { Cookie: p.cookie } : {}), ...headers },
|
||||
body: body !== undefined ? JSON.stringify(body) : undefined,
|
||||
});
|
||||
const text = await res.text();
|
||||
let json: any;
|
||||
try {
|
||||
json = JSON.parse(text);
|
||||
} catch {
|
||||
json = text;
|
||||
}
|
||||
return { status: res.status, json };
|
||||
}
|
||||
|
||||
async function person(name: string): Promise<Person> {
|
||||
const res = await fetch(base + "/api/auth/sign-up/email", {
|
||||
method: "POST",
|
||||
headers: { Origin: webURL, "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ email: `${name}@example.test`, password: "a long password", name, username: name }),
|
||||
});
|
||||
return { cookie: res.headers.getSetCookie().map((c) => c.split(";")[0]).join("; "), name };
|
||||
}
|
||||
|
||||
const file = (name: string, n: number) => ({ name, sha256: n.toString(16).padStart(64, "0"), size: 100 * n });
|
||||
|
||||
/** A person's catalog, registered and approved; its key's Authorization header. */
|
||||
async function catalogOf(owner: Person, admin: Person, name: string, membership = "open") {
|
||||
const reg = await call(owner, "POST", "/api/v1/catalogs", {
|
||||
name,
|
||||
url: `https://${name.toLowerCase().replace(/\W/g, "")}.example.test`,
|
||||
contact: "[email protected]",
|
||||
visibility: "public",
|
||||
membership,
|
||||
});
|
||||
expect(reg.status).toBe(201);
|
||||
const id = reg.json.catalog.id as string;
|
||||
expect((await call(admin, "POST", `/api/v1/console/catalogs/${id}/approve`)).status).toBe(200);
|
||||
return { id, auth: { Authorization: `Catalog ${id}:${reg.json.key}` } };
|
||||
}
|
||||
|
||||
/** A product uploaded by a person to a catalog; its ID. */
|
||||
async function upload(maker: Person, c: { id: string; auth: Record<string, string> }, name: string, n: number, rating = "general") {
|
||||
const tok = await call(maker, "POST", "/api/v1/products/upload", { catalog: c.id });
|
||||
expect(tok.status).toBe(200);
|
||||
const r = await call(
|
||||
undefined,
|
||||
"POST",
|
||||
"/api/v1/catalog/revisions",
|
||||
{ uploadToken: tok.json.token, files: [file("index.xml", n)], rating, parent: 0, name, notes: "" },
|
||||
c.auth,
|
||||
);
|
||||
expect(r.status).toBe(201);
|
||||
expect((await call(maker, "POST", `/api/v1/products/${r.json.product}/publish`)).status).toBe(200);
|
||||
return r.json.product as string;
|
||||
}
|
||||
|
||||
|
||||
/** A revision with these files, by a person, to a catalog: the answer. */
|
||||
async function uploadFiles(maker: Person, c: { id: string; auth: Record<string, string> }, name: string, files: unknown[]) {
|
||||
const tok = await call(maker, "POST", "/api/v1/products/upload", { catalog: c.id });
|
||||
return call(undefined, "POST", "/api/v1/catalog/revisions", { uploadToken: tok.json.token, files, rating: "general", parent: 0, name, notes: "" }, c.auth);
|
||||
}
|
||||
|
||||
let admin: Person;
|
||||
let mod: Person;
|
||||
|
||||
test("the first account is the admin; a moderator", async () => {
|
||||
admin = await person("admin1");
|
||||
mod = await person("warden");
|
||||
const id = (await call(mod, "GET", "/api/v1/me")).json.person.id;
|
||||
expect((await call(admin, "POST", `/api/v1/console/people/${id}/role`, { role: "moderator" })).status).toBe(200);
|
||||
});
|
||||
|
||||
test("blocking a file removes the products using it, tells their catalogs, and refuses it in uploads", async () => {
|
||||
const owner = await person("shelfowner");
|
||||
const maker = await person("maker");
|
||||
const copier = await person("copier");
|
||||
const c = await catalogOf(owner, admin, "Block Shelf");
|
||||
const other = await catalogOf(copier, admin, "Copy Shelf");
|
||||
const logo = file("logo.png", 2).sha256;
|
||||
|
||||
const tee = await upload(maker, c, "Logo Tee", 1);
|
||||
const second = await call(maker, "POST", "/api/v1/products/upload", { catalog: c.id, product: tee });
|
||||
expect(
|
||||
(await call(undefined, "POST", "/api/v1/catalog/revisions", { uploadToken: second.json.token, files: [file("index.xml", 1), file("logo.png", 2)], rating: "general", parent: 0, name: "", notes: "Added the logo" }, c.auth)).status,
|
||||
).toBe(201);
|
||||
const copy = (await uploadFiles(copier, other, "Copy Tee", [file("index.xml", 5), file("logo.png", 2)])).json.product as string;
|
||||
const plain = await upload(maker, c, "Plain Tee", 3);
|
||||
|
||||
// Only those who can remove products hub-wide.
|
||||
for (const p of [owner, maker]) {
|
||||
expect((await call(p, "GET", "/api/v1/blocks")).status).toBe(403);
|
||||
expect((await call(p, "GET", `/api/v1/blocks/${logo}`)).status).toBe(403);
|
||||
expect((await call(p, "PUT", `/api/v1/blocks/${logo}`, { reason: "x" })).status).toBe(403);
|
||||
}
|
||||
expect((await call(owner, "GET", "/api/v1/console")).json.sections).not.toContain("blocks");
|
||||
expect((await call(mod, "GET", "/api/v1/console")).json.sections).toContain("blocks");
|
||||
expect((await call(mod, "GET", "/api/v1/blocks/not-a-hash")).status).toBe(400);
|
||||
|
||||
// What blocking it would remove.
|
||||
const preview = (await call(mod, "GET", `/api/v1/blocks/${logo}`)).json;
|
||||
expect(preview.block).toBeNull();
|
||||
expect(preview.products.map((p: any) => [p.name, p.creatorUsername, p.revisions, p.removed]).sort()).toEqual([
|
||||
["Copy Tee", "copier", [1], false],
|
||||
["Logo Tee", "maker", [2], false],
|
||||
]);
|
||||
|
||||
const blocked = await call(mod, "PUT", `/api/v1/blocks/${logo}`, { reason: "Trademarked logo" });
|
||||
expect(blocked.status).toBe(200);
|
||||
expect(blocked.json.removed.map((p: any) => p.id).sort()).toEqual([copy, tee].sort());
|
||||
expect((await call(mod, "PUT", `/api/v1/blocks/${logo}`, { reason: "Again" })).json.error).toBe("blocked");
|
||||
expect((await call(undefined, "GET", `/api/v1/products/${tee}`)).status).toBe(410);
|
||||
expect((await call(undefined, "GET", `/api/v1/products/${copy}`)).status).toBe(410);
|
||||
expect((await call(undefined, "GET", `/api/v1/products/${plain}`)).status).toBe(200);
|
||||
expect((await call(mod, "GET", `/api/v1/blocks/${logo}`)).json.block).toMatchObject({ sha256: logo, reason: "Trademarked logo", by: "warden" });
|
||||
expect((await call(mod, "GET", "/api/v1/blocks")).json.blocks.map((b: any) => b.sha256)).toEqual([logo]);
|
||||
|
||||
// Each catalog that hosts it is told to delete it.
|
||||
for (const k of [c, other]) {
|
||||
const feed = (await call(undefined, "GET", "/api/v1/catalog/changes", undefined, k.auth)).json.changes;
|
||||
expect(feed.filter((x: any) => x.type === "block")).toEqual([{ type: "block", sha256: logo }]);
|
||||
}
|
||||
|
||||
// Creators are told, and may appeal the removal.
|
||||
const told = (await call(maker, "GET", "/api/v1/notifications")).json.notifications[0];
|
||||
expect(told).toMatchObject({ type: "product_removed", data: { product: tee, reason: "Trademarked logo" } });
|
||||
const removal = (await call(maker, "GET", `/api/v1/console/audit?action=product.remove`)).json.entries[0];
|
||||
expect(removal).toMatchObject({ target: tee, appealable: true, detail: { file: logo } });
|
||||
|
||||
// Uploads with it are refused, and moderators see who tried.
|
||||
const again = await uploadFiles(copier, other, "Copy Tee 2", [file("index.xml", 6), file("logo2.png", 2)]);
|
||||
expect(again.status).toBe(409);
|
||||
expect(again.json.error).toBe("blocked");
|
||||
const tried = (await call(mod, "GET", "/api/v1/console/audit?action=file.blocked_upload")).json.entries[0];
|
||||
expect(tried).toMatchObject({ actor: "copier", tags: { catalog: other.id }, detail: { files: [{ name: "logo2.png", sha256: logo }] } });
|
||||
|
||||
// Unblocking lets it in again; the products it removed stay removed until restored.
|
||||
expect((await call(owner, "DELETE", `/api/v1/blocks/${logo}`)).status).toBe(403);
|
||||
expect((await call(mod, "DELETE", `/api/v1/blocks/${logo}`, { reason: "Licensed after all" })).status).toBe(204);
|
||||
expect((await call(mod, "DELETE", `/api/v1/blocks/${logo}`)).status).toBe(404);
|
||||
expect((await uploadFiles(copier, other, "Copy Tee 2", [file("index.xml", 6), file("logo2.png", 2)])).status).toBe(201);
|
||||
expect((await call(undefined, "GET", `/api/v1/products/${tee}`)).status).toBe(410);
|
||||
expect((await call(mod, "POST", `/api/v1/products/${tee}/unremove`, { reason: "Licensed" })).status).toBe(200);
|
||||
expect((await call(undefined, "GET", `/api/v1/products/${tee}`)).status).toBe(200);
|
||||
expect((await call(mod, "GET", "/api/v1/console/audit?action=file.unblock")).json.entries[0]).toMatchObject({ target: logo, detail: { reason: "Licensed after all" } });
|
||||
});
|
||||
@@ -300,7 +300,7 @@ test("the console is for everyone who controls something; its audit log shows on
|
||||
expect(ownerConsole.sections).toEqual(["overview", "catalogs", "cases", "audit"]);
|
||||
expect(ownerConsole.moderation).toBeNull();
|
||||
expect((await call(maker, "GET", "/api/v1/console")).json.sections).toEqual(["overview", "products", "cases", "audit"]);
|
||||
expect((await call(admin, "GET", "/api/v1/console")).json.sections).toEqual(["overview", "products", "catalogs", "cases", "people", "audit", "settings"]);
|
||||
expect((await call(admin, "GET", "/api/v1/console")).json.sections).toEqual(["overview", "products", "catalogs", "cases", "people", "blocks", "audit", "settings"]);
|
||||
|
||||
// The owner sees their catalog's entries, not the other catalog's; the maker sees their product's.
|
||||
const ownerLog = (await call(owner, "GET", "/api/v1/console/audit")).json.entries;
|
||||
|
||||
Reference in new issue
Block a user