t5de-renderer/docs/interface.md

IMVU Classic's renderer interface

Findings from client 554.8, from IMVU's exported symbols and Ghidra (headless). Ghidra projects hold IMVU's code: keep them out of this repository (%LOCALAPPDATA%\t5de-dev\ghidra; Ghidra also refuses paths with a dot-folder). Our declarations are in include/t5de/imvu/.

How the client creates a renderer

Settings > Advanced > Graphics offers three renderers. The Python setting graphicsRenderer maps to a RendererType, passed to SceneWindow.dll's createRenderer(boost::shared_ptr<INativeWindow>, RendererType):

Setting	Type	Renderer
Direct3D	1	RenderEngineDx, Direct3D 9, built into SceneWindow.dll
OpenGL	2	a plugin, glRenderEngine.dll (RenderEngineGl), via createDLLRenderer
Software ("Standard graphics")	other	RenderEngineDx on SwiftShader (swiftshader.dll, Direct3D 9 on the CPU)

D3D11 and OpenGLSM2 exist only as Python enum values.

createDLLRenderer(window, const wchar_t* dll) (exported) loads a renderer plugin: LoadLibraryW(dll), then the DLL's isRenderEngineSupported() (nonzero to accept), then createRenderEngine(window). For type 2, createRenderer passes L"glRenderEngine.dll" with a single push of the string's address, so a T5DE patch can point that operand at our DLL's name in memory (after checking the instruction) to load this plugin instead.

The plugin's exports:

• int isRenderEngineSupported(): IMVU's OpenGL plugin returns 1.
• boost::shared_ptr<PiRenderEngine> createRenderEngine(boost::shared_ptr<INativeWindow> window): __cdecl; the result through a hidden pointer; the callee destroys the by-value argument (MSVC), releasing its reference.

PiRenderEngine

A 2006-era fixed-function interface: 35 virtual methods (__thiscall). Names come from RenderEngineDx's exported methods; signatures from their decorated names. include/t5de/imvu/render_engine.hpp declares them in vtable order.

• Frame: beginRendering(PiRenderTarget*), endRendering(), clear(flags, color), resizeFrameBuffer(Vec2<int>), enable2dFrameBuffer(), render targets (newPixmapRenderTarget, newTextureRenderTarget, returning boost::shared_ptrs).
• Transforms: setProjection(Vec2<int>, const Mat4f&), setViewSpace and setModelSpace(const Transform&), texture transforms per stage.
• State: depth test and write, blend mode and blending, alpha test and value, back-face culling, normal inverting, lighting, setLightProperties(index, const LightProperties&), fog, environment ambient color.
• Resources: newRenderTexture(Pixmap*, flags), index and vertex buffers.
• Drawing: drawTriangles (an index buffer and four vertex buffers), drawLine, drawPointSprites, drawTrianglesFromMemory (vertex attribute arrays).
• Two textures per draw (setTextures(texture0, texture1)).

SceneWindow.dll defines the base class: its constructor and destructor, defaults for the buffer methods and drawTriangles (IMVU's OpenGL plugin uses them), and an asserting enable2dFrameBuffer. Its fields, after the vtable pointer: a magic number (0x3b913159, checked by the destructor) and a size counter (12 bytes in all). Outside the virtual methods, IMVU's code only uses an engine through helpers that call its virtual methods (render states' apply, resources' allocateInternalResource).

ABI

• 32-bit; virtual methods are __thiscall. Modern MSVC and clang-cl lay out single-inheritance vtables and pass arguments the same way as Visual C++ 2010, which built the client.
• boost::shared_ptr (Boost 1.61): the object pointer, then the control block: a vtable (destructor, dispose, destroy, get_deleter, get_untyped_deleter), the use count and the weak count, changed atomically. include/t5de/imvu/shared_ptr.hpp matches it. With non-trivial copy and destruction, it's returned through a hidden pointer and passed on the stack, as IMVU's code expects.
• The data types are in include/t5de/imvu/types.hpp, worked out from the Direct3D engine: matrices and transforms in Direct3D's row-vector convention, colors as 0xAARRGGBB, lights (point or directional), blend modes, linear fog, texture coordinate transforms, vertex attribute arrays (data, count, stride). Pixmap is used through pixmap.dll's exported methods (width, height, pitch, pixels, ...), not its layout.
• drawTriangles' vertex buffers are positions (Vec3f), normals (Vec3f), texture coordinates (Vec2f) and colors; vertex buffers are made with a stride, element size and count, dynamic ones filled later through the buffer's own update. The objects an engine creates (buffers, textures, render targets) are declared in include/t5de/imvu/resources.hpp; IMVU's code deletes them through their virtual destructors.
• Checked in the client with the pass-through's logged arguments: a right-handed perspective projection (45 degree vertical field of view, Direct3D's 0 to 1 depth) and an orthographic one for 2D, orthonormal view rotations, opaque clear colors, ambient light with zero alpha, linear fog (1 to 50), scrolling texture transforms. Furniture vertices are interleaved: positions and normals are 12-byte elements 32 bytes apart, so strides matter.
• Projections give depth from -1 to 1, as OpenGL's do: IMVU's Direct3D renderer multiplies them by a matrix mapping it to 0 to 1 (z' = z / 2 + w / 2) and shifting them half a pixel (-1 / width, +1 / height, for Direct3D 9's pixel centers). Without the depth mapping, everything in the nearer half of the depth range is clipped: the 2D overlay (name tags, the frame rate, the room's title) is drawn at depth -0.98.
• The 2D overlay is ordinary drawTriangles with an orthographic projection (a few per frame), textured with text IMVU draws with GDI into bitmaps.
• Particles (drawPointSprites): a Visual C++ 2010 std::vector of 16-byte points (position, color), drawn as Direct3D 9 point sprites: squares viewport height / distance pixels across (IMVU sets the size, 1, and the scale factors 0, 0, 1), textured over the whole square. Direct3D 9's documented 64-pixel default limit doesn't apply in practice: clamped to it they look much smaller and fainter than the client's.
• Fog: linear table fog, start 1 and end 50 in rooms. Fogging by view depth matches the client; by depth buffer value (as Direct3D 9's documentation suggests for projections like IMVU's) doesn't.
• Render targets (include/t5de/imvu/resources.hpp): the engine returns each as a pointer into its own object; the client calls only their interfaces' methods and passes them to beginRendering. A pixmap render target has one method, getPixmap() (a Pixmap returned by value), with the pixels of the last frame drawn into it (IMVU's engine copies them from the GPU at the frame's end). A texture render target has area() (always 0, 0, 1, 1) and texture() (a new texture object for what's been drawn, as a shared pointer returned by value). The client uses pixmap targets for snapshots (takeSnapshot: outfit pictures, the photo widget), 2048x2048; texture targets haven't been seen used. Pixmap is 28 bytes; pixmap.dll exports its constructors and destructor.
• Still unknown: CompoundRegion (texture update regions), newRenderTexture's flags, and what beginRendering does with a null target. MSVC orders overloaded virtual methods unlike their declaration, so our declarations never overload virtual methods.
• Memory crosses the boundary only through virtual methods (destructors, control blocks), so each side can use its own C runtime. Exceptions must not cross it.

What a renderer can do

• Every draw passes through the interface between beginRendering and endRendering, so a renderer can record a frame and draw it however it likes: shaders emulating the fixed-function state, per-pixel lighting, anti-aliasing, anisotropic filtering, correct gamma and tone mapping, shadows and screen-space effects from the recorded frame, post-processing.
• It receives only two textures, a few lights, fog and blend state per draw. Newer material features (normal, roughness, emissive maps) aren't passed: they'd need hooks where SceneWindow.dll loads products, and the maps must exist in the product data the servers send Classic (unverified).
• Frame rate is not the renderer's: the scene advances in 33 ms ticks, and animations one frame per tick (ActionInstance::step calls incrementActionFrame per tick; FrameIntervalMaker::getFrameInterval counts the ticks; runApp waits out zero.dll's millisecondsPerFrame). Shorter ticks speed animations up. Smooth high frame rates need in-between poses: hooks in SceneWindow.dll's animation update.

What the client draws

From the pass-through (Direct3D, client 554.8, a few rooms):

• A room frame: about 146 drawTriangles, each with its own setModelSpace (131) and setTextures (138), with per-object alpha test, culling, normal inverting and blending, 252 texture-transform resets, 8 light updates; no drawLine, drawPointSprites or drawTrianglesFromMemory in those rooms.
• Every frame creates index and vertex buffers (4 and 12 in a room, with nothing new loading): most likely avatars, skinned on the CPU and uploaded again each frame.
• Each 3D window, and each room change, creates a new engine (createRenderEngine), so engines start and stop often; frame buffers are resized freely (up to 3440x1272 here).
• IMVU's OpenGL engine needs a real graphics driver (ARB_multitexture); on Windows' basic display driver (virtual machines) it throws and the client falls back. Its exceptions are std::exceptions our code can catch.

Plan

1. Pass-through plugin (done, verified in rooms): wraps IMVU's Direct3D engine, made with SceneWindow.dll's exported createRenderer, and logs what each frame calls, proving our declarations.
2. Work out the layouts above.
3. A renderer reproducing the current look on Direct3D 11 (it also runs in software, on WARP, where OpenGL can't), then the improvements.