t5de-renderer/docs/interface.md

# IMVU Classic's renderer interface

Findings from client 554.8, from IMVU's exported symbols and Ghidra (headless). Ghidra projects hold IMVU's code:
keep them out of this repository (`%LOCALAPPDATA%\t5de-dev\ghidra`; Ghidra also refuses paths with a dot-folder).
Our declarations are in `include/t5de/imvu/`.

## How the client creates a renderer

Settings > Advanced > Graphics offers three renderers. The Python setting `graphicsRenderer` maps to a
`RendererType`, passed to `SceneWindow.dll`'s `createRenderer(boost::shared_ptr<INativeWindow>, RendererType)`:

| Setting | Type | Renderer |
|---|---|---|
| Direct3D | 1 | `RenderEngineDx`, Direct3D 9, built into `SceneWindow.dll` |
| OpenGL | 2 | a plugin, `glRenderEngine.dll` (`RenderEngineGl`), via `createDLLRenderer` |
| Software ("Standard graphics") | other | `RenderEngineDx` on SwiftShader (`swiftshader.dll`, Direct3D 9 on the CPU) |

`D3D11` and `OpenGLSM2` exist only as Python enum values.

`createDLLRenderer(window, const wchar_t* dll)` (exported) loads a renderer plugin: `LoadLibraryW(dll)`, then the
DLL's `isRenderEngineSupported()` (nonzero to accept), then `createRenderEngine(window)`. For type 2,
`createRenderer` passes `L"glRenderEngine.dll"` with a single `push` of the string's address, so a T5DE patch can
point that operand at our DLL's name in memory (after checking the instruction) to load this plugin instead.

The plugin's exports:

- `int isRenderEngineSupported()`: IMVU's OpenGL plugin returns 1.
- `boost::shared_ptr<PiRenderEngine> createRenderEngine(boost::shared_ptr<INativeWindow> window)`: `__cdecl`; the
  result through a hidden pointer; the callee destroys the by-value argument (MSVC), releasing its reference.

## `PiRenderEngine`

A 2006-era fixed-function interface: 35 virtual methods (`__thiscall`). Names come from `RenderEngineDx`'s exported
methods; signatures from their decorated names. `include/t5de/imvu/render_engine.hpp` declares them in vtable order.

- Frame: `beginRendering(PiRenderTarget*)`, `endRendering()`, `clear(flags, color)`, `resizeFrameBuffer(Vec2<int>)`,
  `enable2dFrameBuffer()`, render targets (`newPixmapRenderTarget`, `newTextureRenderTarget`, returning
  `boost::shared_ptr`s).
- Transforms: `setProjection(Vec2<int>, const Mat4f&)`, `setViewSpace` and `setModelSpace(const Transform&)`,
  texture transforms per stage.
- State: depth test and write, blend mode and blending, alpha test and value, back-face culling, normal inverting,
  lighting, `setLightProperties(index, const LightProperties&)`, fog, environment ambient color.
- Resources: `newRenderTexture(Pixmap*, flags)`, index and vertex buffers.
- Drawing: `drawTriangles` (an index buffer and four vertex buffers), `drawLine`, `drawPointSprites`,
  `drawTrianglesFromMemory` (vertex attribute arrays).
- Two textures per draw (`setTextures(texture0, texture1)`).

`SceneWindow.dll` defines the base class: its constructor and destructor, defaults for the buffer methods and
`drawTriangles` (IMVU's OpenGL plugin uses them), and an asserting `enable2dFrameBuffer`. Its fields, after the vtable
pointer: a magic number (`0x3b913159`, checked by the destructor) and a size counter (12 bytes in all). Outside the
virtual methods, IMVU's code only uses an engine through helpers that call its virtual methods (render states'
`apply`, resources' `allocateInternalResource`).

## ABI

- 32-bit; virtual methods are `__thiscall`. Modern MSVC and clang-cl lay out single-inheritance vtables and pass
  arguments the same way as Visual C++ 2010, which built the client.
- `boost::shared_ptr` (Boost 1.61): the object pointer, then the control block: a vtable (destructor, `dispose`,
  `destroy`, `get_deleter`, `get_untyped_deleter`), the use count and the weak count, changed atomically.
  `include/t5de/imvu/shared_ptr.hpp` matches it. With non-trivial copy and destruction, it's returned through a
  hidden pointer and passed on the stack, as IMVU's code expects.
- The data types are in `include/t5de/imvu/types.hpp`, worked out from the Direct3D engine: matrices and
  transforms in Direct3D's row-vector convention, colors as `0xAARRGGBB`, lights (point or directional), blend
  modes, linear fog, texture coordinate transforms, vertex attribute arrays (data, count, stride). `Pixmap` is used
  through `pixmap.dll`'s exported methods (`width`, `height`, `pitch`, `pixels`, ...), not its layout.
- `drawTriangles`' vertex buffers are positions (`Vec3f`), normals (`Vec3f`), texture coordinates (`Vec2f`) and
  colors; vertex buffers are made with a stride, element size and count, dynamic ones filled later through the
  buffer's own `update`. The objects an engine creates (buffers, textures, render targets) are declared in
  `include/t5de/imvu/resources.hpp`; IMVU's code deletes them through their virtual destructors.
- Checked in the client with the pass-through's logged arguments: a right-handed perspective projection (45 degree
  vertical field of view, Direct3D's 0 to 1 depth) and an orthographic one for 2D, orthonormal view rotations,
  opaque clear colors, ambient light with zero alpha, linear fog (1 to 50), scrolling texture transforms. Furniture
  vertices are interleaved: positions and normals are 12-byte elements 32 bytes apart, so strides matter.
- Projections give depth from -1 to 1, as OpenGL's do: IMVU's Direct3D renderer multiplies them by a matrix
  mapping it to 0 to 1 (z' = z / 2 + w / 2) and shifting them half a pixel (-1 / width, +1 / height, for Direct3D
  9's pixel centers). Without the depth mapping, everything in the nearer half of the depth range is clipped: the
  2D overlay (name tags, the frame rate, the room's title) is drawn at depth -0.98.
- The 2D overlay is ordinary `drawTriangles` with an orthographic projection (a few per frame), textured with text
  IMVU draws with GDI into bitmaps.
- Particles (`drawPointSprites`): a Visual C++ 2010 `std::vector` of 16-byte points (position, color), drawn as
  Direct3D 9 point sprites: squares viewport height / distance pixels across (IMVU sets the size, 1, and the scale
  factors 0, 0, 1), textured over the whole square. Direct3D 9's documented 64-pixel default limit doesn't apply in
  practice: clamped to it they look much smaller and fainter than the client's.
- Fog: linear table fog, start 1 and end 50 in rooms. Fogging by view depth matches the client; by depth buffer
  value (as Direct3D 9's documentation suggests for projections like IMVU's) doesn't.
- Still unknown: render targets, `CompoundRegion` (texture update
  regions), `newRenderTexture`'s flags, and what `beginRendering` does with a null target. MSVC orders overloaded
  virtual methods unlike their declaration, so our declarations never overload virtual methods.
- Memory crosses the boundary only through virtual methods (destructors, control blocks), so each side can use its
  own C runtime. Exceptions must not cross it.

## What a renderer can do

- Every draw passes through the interface between `beginRendering` and `endRendering`, so a renderer can record a
  frame and draw it however it likes: shaders emulating the fixed-function state, per-pixel lighting,
  anti-aliasing, anisotropic filtering, correct gamma and tone mapping, shadows and screen-space effects from the
  recorded frame, post-processing.
- It receives only two textures, a few lights, fog and blend state per draw. Newer material features (normal,
  roughness, emissive maps) aren't passed: they'd need hooks where `SceneWindow.dll` loads products, and the maps
  must exist in the product data the servers send Classic (unverified).
- Frame rate is not the renderer's: the scene advances in 33 ms ticks, and animations one frame per tick
  (`ActionInstance::step` calls `incrementActionFrame` per tick; `FrameIntervalMaker::getFrameInterval` counts the
  ticks; `runApp` waits out `zero.dll`'s `millisecondsPerFrame`). Shorter ticks speed animations up. Smooth high
  frame rates need in-between poses: hooks in `SceneWindow.dll`'s animation update.

## What the client draws

From the pass-through (Direct3D, client 554.8, a few rooms):

- A room frame: about 146 `drawTriangles`, each with its own `setModelSpace` (131) and `setTextures` (138), with
  per-object alpha test, culling, normal inverting and blending, 252 texture-transform resets, 8 light updates; no
  `drawLine`, `drawPointSprites` or `drawTrianglesFromMemory` in those rooms.
- Every frame creates index and vertex buffers (4 and 12 in a room, with nothing new loading): most likely avatars,
  skinned on the CPU and uploaded again each frame.
- Each 3D window, and each room change, creates a new engine (`createRenderEngine`), so engines start and stop
  often; frame buffers are resized freely (up to 3440x1272 here).
- IMVU's OpenGL engine needs a real graphics driver (`ARB_multitexture`); on Windows' basic display driver
  (virtual machines) it throws and the client falls back. Its exceptions are `std::exception`s our code can
  catch.

## Plan

1. Pass-through plugin (done, verified in rooms): wraps IMVU's Direct3D engine, made with `SceneWindow.dll`'s
   exported `createRenderer`, and logs what each frame calls, proving our declarations.
2. Work out the layouts above.
3. A renderer reproducing the current look on Direct3D 11 (it also runs in software, on WARP, where OpenGL can't),
   then the improvements.