t5de-renderer/docs/interface.md

IMVU Classic's renderer interface

Findings from client 554.8, from IMVU's exported symbols and Ghidra (headless). Ghidra projects hold IMVU's code: keep them out of this repository (%LOCALAPPDATA%\t5de-dev\ghidra; Ghidra also refuses paths with a dot-folder). Our declarations are in include/t5de/imvu/.

How the client creates a renderer

Settings > Advanced > Graphics offers three renderers. The Python setting graphicsRenderer maps to a RendererType, passed to SceneWindow.dll's createRenderer(boost::shared_ptr<INativeWindow>, RendererType):

Setting	Type	Renderer
Direct3D	1	RenderEngineDx, Direct3D 9, built into SceneWindow.dll
OpenGL	2	a plugin, glRenderEngine.dll (RenderEngineGl), via createDLLRenderer
Software ("Standard graphics")	other	RenderEngineDx on SwiftShader (swiftshader.dll, Direct3D 9 on the CPU)

D3D11 and OpenGLSM2 exist only as Python enum values.

createDLLRenderer(window, const wchar_t* dll) (exported) loads a renderer plugin: LoadLibraryW(dll), then the DLL's isRenderEngineSupported() (nonzero to accept), then createRenderEngine(window). For type 2, createRenderer passes L"glRenderEngine.dll" with a single push of the string's address, so a T5DE patch can point that operand at our DLL's name in memory (after checking the instruction) to load this plugin instead.

The plugin's exports:

• int isRenderEngineSupported(): IMVU's OpenGL plugin returns 1.
• boost::shared_ptr<PiRenderEngine> createRenderEngine(boost::shared_ptr<INativeWindow> window): __cdecl; the result through a hidden pointer; the callee destroys the by-value argument (MSVC), releasing its reference.

PiRenderEngine

A 2006-era fixed-function interface: 35 virtual methods (__thiscall). Names come from RenderEngineDx's exported methods; signatures from their decorated names. include/t5de/imvu/render_engine.hpp declares them in vtable order.

• Frame: beginRendering(PiRenderTarget*), endRendering(), clear(flags, color), resizeFrameBuffer(Vec2<int>), enable2dFrameBuffer(), render targets (newPixmapRenderTarget, newTextureRenderTarget, returning boost::shared_ptrs).
• Transforms: setProjection(Vec2<int>, const Mat4f&), setViewSpace and setModelSpace(const Transform&), texture transforms per stage.
• State: depth test and write, blend mode and blending, alpha test and value, back-face culling, normal inverting, lighting, setLightProperties(index, const LightProperties&), fog, environment ambient color.
• Resources: newRenderTexture(Pixmap*, flags), index and vertex buffers.
• Drawing: drawTriangles (an index buffer and four vertex buffers), drawLine, drawPointSprites, drawTrianglesFromMemory (vertex attribute arrays).
• Two textures per draw (setTextures(texture0, texture1)).

SceneWindow.dll defines the base class: its constructor and destructor, defaults for the buffer methods and drawTriangles (IMVU's OpenGL plugin uses them), and an asserting enable2dFrameBuffer. Its fields, after the vtable pointer: a magic number (0x3b913159, checked by the destructor) and a size counter (12 bytes in all). Outside the virtual methods, IMVU's code only uses an engine through helpers that call its virtual methods (render states' apply, resources' allocateInternalResource).

ABI

• 32-bit; virtual methods are __thiscall. Modern MSVC and clang-cl lay out single-inheritance vtables and pass arguments the same way as Visual C++ 2010, which built the client.
• boost::shared_ptr (Boost 1.61): the object pointer, then the control block: a vtable (destructor, dispose, destroy, get_deleter, get_untyped_deleter), the use count and the weak count, changed atomically. include/t5de/imvu/shared_ptr.hpp matches it. With non-trivial copy and destruction, it's returned through a hidden pointer and passed on the stack, as IMVU's code expects.
• The data types are in include/t5de/imvu/types.hpp, worked out from the Direct3D engine: matrices and transforms in Direct3D's row-vector convention, colors as 0xAARRGGBB, lights (point or directional), blend modes, linear fog, texture coordinate transforms, vertex attribute arrays (data, count, stride). Pixmap is used through pixmap.dll's exported methods (width, height, pitch, pixels, ...), not its layout.
• drawTriangles' vertex buffers are positions (Vec3f), normals (Vec3f), texture coordinates (Vec2f) and colors; vertex buffers are made with a stride, element size and count, dynamic ones filled later through the buffer's own update. The objects an engine creates (buffers, textures, render targets) are declared in include/t5de/imvu/resources.hpp; IMVU's code deletes them through their virtual destructors.
• Checked in the client with the pass-through's logged arguments: a right-handed perspective projection (45 degree vertical field of view, Direct3D's 0 to 1 depth) and an orthographic one for 2D, orthonormal view rotations, opaque clear colors, ambient light with zero alpha, linear fog (1 to 50), scrolling texture transforms. Furniture vertices are interleaved: positions and normals are 12-byte elements 32 bytes apart, so strides matter.
• Particles (drawPointSprites): a Visual C++ 2010 std::vector of 16-byte points (position, color), drawn as Direct3D 9 point sprites: squares viewport height / distance pixels across (Direct3D 9's defaults clamp them to 1 to 64; IMVU sets only the size, 1, and the scale factors 0, 0, 1), textured over the whole square.
• Fog: linear table fog, start 1 and end 50 in rooms. Fogging by view depth matches the client; by depth buffer value (as Direct3D 9's documentation suggests for projections like IMVU's) doesn't.
• Still unknown: render targets, CompoundRegion (texture update regions), newRenderTexture's flags, and what beginRendering does with a null target. MSVC orders overloaded virtual methods unlike their declaration, so our declarations never overload virtual methods.
• Memory crosses the boundary only through virtual methods (destructors, control blocks), so each side can use its own C runtime. Exceptions must not cross it.

What a renderer can do

• Every draw passes through the interface between beginRendering and endRendering, so a renderer can record a frame and draw it however it likes: shaders emulating the fixed-function state, per-pixel lighting, anti-aliasing, anisotropic filtering, correct gamma and tone mapping, shadows and screen-space effects from the recorded frame, post-processing.
• It receives only two textures, a few lights, fog and blend state per draw. Newer material features (normal, roughness, emissive maps) aren't passed: they'd need hooks where SceneWindow.dll loads products, and the maps must exist in the product data the servers send Classic (unverified).
• Frame rate is not the renderer's: the scene advances in 33 ms ticks, and animations one frame per tick (ActionInstance::step calls incrementActionFrame per tick; FrameIntervalMaker::getFrameInterval counts the ticks; runApp waits out zero.dll's millisecondsPerFrame). Shorter ticks speed animations up. Smooth high frame rates need in-between poses: hooks in SceneWindow.dll's animation update.

What the client draws

From the pass-through (Direct3D, client 554.8, a few rooms):

• A room frame: about 146 drawTriangles, each with its own setModelSpace (131) and setTextures (138), with per-object alpha test, culling, normal inverting and blending, 252 texture-transform resets, 8 light updates; no drawLine, drawPointSprites or drawTrianglesFromMemory in those rooms.
• Every frame creates index and vertex buffers (4 and 12 in a room, with nothing new loading): most likely avatars, skinned on the CPU and uploaded again each frame.
• Each 3D window, and each room change, creates a new engine (createRenderEngine), so engines start and stop often; frame buffers are resized freely (up to 3440x1272 here).
• IMVU's OpenGL engine needs a real graphics driver (ARB_multitexture); on Windows' basic display driver (virtual machines) it throws and the client falls back. Its exceptions are std::exceptions our code can catch.

Plan

1. Pass-through plugin (done, verified in rooms): wraps IMVU's Direct3D engine, made with SceneWindow.dll's exported createRenderer, and logs what each frame calls, proving our declarations.
2. Work out the layouts above.
3. A renderer reproducing the current look on Direct3D 11 (it also runs in software, on WARP, where OpenGL can't), then the improvements.