t5de-renderer/docs/architecture.md

Architecture

The renderer is two programs:

• The adapter, t5deRenderEngine.dll (32-bit, adapter/): a renderer plugin IMVU Classic loads in place of its OpenGL renderer (selected by the t5de-renderer patch in the t5de repository). It implements IMVU's PiRenderEngine interface (include/t5de/imvu/, docs/interface.md) and passes what the client draws to the renderer process. It stays thin: IMVU's process is 32-bit (4 GB of address space shared with Python, Gecko and IMVU's own code) and tied to Visual C++ 2010's ABI.
• The renderer process, t5de-renderer.exe (64-bit, renderer/): draws, with Diligent Engine (Apache-2.0). Being 64-bit and separate, it can use any modern API and library, isn't limited by the client's address space, and can crash without taking the client down.

Why Diligent: one API over Direct3D 11 (every Windows PC, and software rendering on machines without a graphics driver), Direct3D 12 and Vulkan (ray tracing, mesh shaders), and WebGPU (browsers, for a possible future client). It's built with CMake and fetched by FetchContent, pinned to a commit (renderer/CMakeLists.txt); only the Direct3D 11 backend is built for now.

How they talk

include/t5de/protocol/ is shared by both builds, so it uses fixed-size fields only. For each engine the client creates (each 3D view, and again on room changes), the adapter:

1. creates a shared memory channel (Local\t5de-renderer-<pid>-<n>) holding IMVU's window handle and a frame count, and an event set after each frame;
2. starts t5de-renderer.exe --channel <name> --parent <pid> from its own folder;
3. sets the event at each endRendering, and sets closing when the engine is destroyed.

The renderer process draws into a child window it creates inside the client's window: window handles are valid in every process, and Windows allows child windows across processes (Chrome's GPU process works this way). The child is disabled so mouse input goes to the client's window under it, and the adapter gives the client's window WS_CLIPCHILDREN so the client's own drawing stays out of it. The process exits when the engine is destroyed or the client exits.

Status

The renderer process draws the client's whole scene, over the right half of the 3D view, beside the client's own Direct3D renderer on the left (the adapter still forwards everything to it). Checked in rooms with screenshots of both renderers side by side (the compare view): they match, particles and the 2D overlay (name tags) included.

For comparing while the client runs (development):

• Views: T5DE_RENDERER_VIEW (off, split, full or compare) when the client starts, or %TEMP%\t5de-renderer-view.txt holding one of those at any time. Compare draws the left half's scene over the right half, so both renderers show the same thing side by side.

• %TEMP%\t5de-renderer-shaders\fixed_function.hlsl, if present, replaces the built-in shaders and is reloaded when it changes.

• The renderer process runs from a copy (%TEMP%\t5de-renderer), so t5de-renderer.exe can be replaced while the client runs; engines started afterwards run the new one (the room's view keeps its engine until the client restarts).

• The adapter's log (%TEMP%\t5de-renderer.log) sums up what the client draws every 300 frames (the first few times), and logs textures and render targets the client renderer made itself.

• The command stream (include/t5de/protocol/commands.hpp) carries state, transforms, lights, fog, textures, buffers (copied when created or updated) and draws, in chunks through the channel; the adapter waits for each to be taken, and gives up on a renderer process that stops taking them (the client's renderer carries on).

• The renderer process reproduces IMVU's Direct3D 9 setup (renderer/shaders/fixed_function.hlsl), particles included (point sprites, drawn as instanced squares), with a pipeline per combination of states.

• Speed: on a PC with a graphics card, a room frame takes about 2 ms to draw (some 430 draws), and the time from the client finishing a frame to ours being presented is about 3 ms: the halves can't be told apart, even while the camera moves. On the development VM (software rendering, over Remote Desktop) drawing takes 8 to 60 ms, and the right half visibly trails, mostly from compositing another process's window over Remote Desktop. The renderer process draws only the newest frame when it falls behind, and keeps at most one frame queued.

Next: render targets (offscreen frames are skipped), the renderer process drawing the whole view in place of the client's renderer, and one renderer process for all of a client's views.

Building

Two presets, one per architecture: cmake --workflow --preset adapter (32-bit, with the tests) and cmake --workflow --preset renderer (64-bit; the first configure fetches and builds Diligent). Both programs go in the client's folder.