dhkatzandClaude Opus 5.5
7bf29f3e4c
CI / build (push) Successful in 54s
The client creates its 3D view's window with it, so not adding it wasn't enough: covered entirely by the renderer's window, the client's window got no WM_PAINT and the client still drew about once a second. Co-Authored-By: Claude Opus 5.5 <[email protected]>
81 lines
5.6 KiB
Markdown
81 lines
5.6 KiB
Markdown
# Architecture
|
|
|
|
The renderer is two programs:
|
|
|
|
- **The adapter**, `t5deRenderEngine.dll` (32-bit, `adapter/`): a renderer plugin IMVU Classic loads in place of its
|
|
OpenGL renderer (selected by the `t5de-renderer` patch in the `t5de` repository). It implements IMVU's
|
|
`PiRenderEngine` interface (`include/t5de/imvu/`, `docs/interface.md`) and passes what the client draws to the
|
|
renderer process. It stays thin: IMVU's process is 32-bit (4 GB of address space shared with Python, Gecko and
|
|
IMVU's own code) and tied to Visual C++ 2010's ABI.
|
|
- **The renderer process**, `t5de-renderer.exe` (64-bit, `renderer/`): draws, with
|
|
[Diligent Engine](https://github.com/DiligentGraphics/DiligentCore) (Apache-2.0). Being 64-bit and separate, it can
|
|
use any modern API and library, isn't limited by the client's address space, and can crash without taking the
|
|
client down.
|
|
|
|
Why Diligent: one API over Direct3D 11 (every Windows PC, and software rendering on machines without a graphics
|
|
driver), Direct3D 12 and Vulkan (ray tracing, mesh shaders), and WebGPU (browsers, for a possible future client).
|
|
It's built with CMake and fetched by `FetchContent`, pinned to a commit (`renderer/CMakeLists.txt`); only the
|
|
Direct3D 11 backend is built for now.
|
|
|
|
## How they talk
|
|
|
|
`include/t5de/protocol/` is shared by both builds, so it uses fixed-size fields only. For each engine the client
|
|
creates (each 3D view, and again on room changes), the adapter:
|
|
|
|
1. creates a shared memory channel (`Local\t5de-renderer-<pid>-<n>`) holding IMVU's window handle and a frame
|
|
count, and an event set after each frame;
|
|
2. starts `t5de-renderer.exe --channel <name> --parent <pid>` from its own folder;
|
|
3. sets the event at each `endRendering`, and sets `closing` when the engine is destroyed.
|
|
|
|
The renderer process draws into a child window it creates inside the client's window: window handles are valid in
|
|
every process, and Windows allows child windows across processes (Chrome's GPU process works this way). The child
|
|
is disabled so mouse input goes to the client's window under it. Its swap chain is a flip model one, which the
|
|
compositor shows above the client's own drawing. The client's window must not clip its children (`WS_CLIPCHILDREN`,
|
|
which the client creates it with, so the adapter clears it):
|
|
the client draws a frame when its window gets `WM_PAINT` (`SceneWindow.dll` invalidates it, and draws on the paint),
|
|
and a window entirely covered by clipped children never gets one, so the client fell to about one frame a second.
|
|
The process exits when the engine is destroyed or the client exits.
|
|
|
|
## Status
|
|
|
|
The renderer process draws the whole 3D view in place of the client's renderer (the full view, the default). The
|
|
adapter still forwards everything to the client's Direct3D renderer (state and resources: it draws render targets and
|
|
reads textures back, which the renderer process doesn't yet), but while the renderer process covers the whole view
|
|
(`Channel::covering`, set after each frame it presents) the client's renderer skips its clears and draws, decided per
|
|
frame; it draws again as soon as the renderer process stops covering the view (the view changes, or the process
|
|
fails). Checked in rooms with screenshots of both renderers side by side (the compare view): they match, particles
|
|
and the 2D overlay (name tags) included.
|
|
|
|
For comparing while the client runs (development):
|
|
|
|
- Views: `T5DE_RENDERER_VIEW` (off, split, full (the default) or compare) when the client starts, or
|
|
`%TEMP%\t5de-renderer-view.txt` holding one of those at any time. Compare draws the left half's scene over the
|
|
right half, so both renderers show the same thing side by side.
|
|
- `%TEMP%\t5de-renderer-shaders\fixed_function.hlsl`, if present, replaces the built-in shaders and is reloaded
|
|
when it changes.
|
|
- The renderer process runs from a copy (`%TEMP%\t5de-renderer`), so `t5de-renderer.exe` can be replaced while the
|
|
client runs; engines started afterwards run the new one (the room's view keeps its engine until the client
|
|
restarts).
|
|
- The adapter's log (`%TEMP%\t5de-renderer.log`) sums up what the client draws every 300 frames (the first few
|
|
times), and logs textures and render targets the client renderer made itself.
|
|
|
|
- The command stream (`include/t5de/protocol/commands.hpp`) carries state, transforms, lights, fog, textures,
|
|
buffers (copied when created or updated) and draws, in chunks through the channel; the adapter waits for each to
|
|
be taken, and gives up on a renderer process that stops taking them (the client's renderer carries on).
|
|
- The renderer process reproduces IMVU's Direct3D 9 setup (`renderer/shaders/fixed_function.hlsl`), particles
|
|
included (point sprites, drawn as instanced squares), with a pipeline per combination of states.
|
|
- Speed: on a PC with a graphics card, a room frame takes about 2 ms to draw (some 430 draws), and the time from
|
|
the client finishing a frame to ours being presented is about 3 ms: the halves can't be told apart, even while
|
|
the camera moves. On the development VM (software rendering, over Remote Desktop) drawing takes 8 to 60 ms, and
|
|
the right half visibly trails, mostly from compositing another process's window over Remote Desktop. The
|
|
renderer process draws only the newest frame when it falls behind, and keeps at most one frame queued.
|
|
|
|
Next: render targets (offscreen frames are skipped, and drawn by the client's renderer), and one renderer process for
|
|
all of a client's views.
|
|
|
|
## Building
|
|
|
|
Two presets, one per architecture: `cmake --workflow --preset adapter` (32-bit, with the tests) and
|
|
`cmake --workflow --preset renderer` (64-bit; the first configure fetches and builds Diligent). Both programs go in
|
|
the client's folder.
|