t5de-renderer/docs/interface.md

IMVU Classic's renderer interface

Findings from client 554.8, from IMVU's exported symbols and Ghidra (headless). Ghidra projects hold IMVU's code: keep them out of this repository (%LOCALAPPDATA%\t5de-dev\ghidra; Ghidra also refuses paths with a dot-folder). Our declarations are in include/t5de/imvu/.

How the client creates a renderer

Settings > Advanced > Graphics offers three renderers. The Python setting graphicsRenderer maps to a RendererType, passed to SceneWindow.dll's createRenderer(boost::shared_ptr<INativeWindow>, RendererType):

Setting	Type	Renderer
Direct3D	1	RenderEngineDx, Direct3D 9, built into SceneWindow.dll
OpenGL	2	a plugin, glRenderEngine.dll (RenderEngineGl), via createDLLRenderer
Software ("Standard graphics")	other	RenderEngineDx on SwiftShader (swiftshader.dll, Direct3D 9 on the CPU)

D3D11 and OpenGLSM2 exist only as Python enum values.

createDLLRenderer(window, const wchar_t* dll) (exported) loads a renderer plugin: LoadLibraryW(dll), then the DLL's isRenderEngineSupported() (nonzero to accept), then createRenderEngine(window). For type 2, createRenderer passes L"glRenderEngine.dll" with a single push of the string's address, so a T5DE patch can point that operand at our DLL's name in memory (after checking the instruction) to load this plugin instead.

The plugin's exports:

• int isRenderEngineSupported(): IMVU's OpenGL plugin returns 1.
• boost::shared_ptr<PiRenderEngine> createRenderEngine(boost::shared_ptr<INativeWindow> window): __cdecl; the result through a hidden pointer; the callee destroys the by-value argument (MSVC), releasing its reference.

PiRenderEngine

A 2006-era fixed-function interface: 35 virtual methods (__thiscall). Names come from RenderEngineDx's exported methods; signatures from their decorated names. include/t5de/imvu/render_engine.hpp declares them in vtable order.

• Frame: beginRendering(PiRenderTarget*), endRendering(), clear(flags, color), resizeFrameBuffer(Vec2<int>), enable2dFrameBuffer(), render targets (newPixmapRenderTarget, newTextureRenderTarget, returning boost::shared_ptrs).
• Transforms: setProjection(Vec2<int>, const Mat4f&), setViewSpace and setModelSpace(const Transform&), texture transforms per stage.
• State: depth test and write, blend mode and blending, alpha test and value, back-face culling, normal inverting, lighting, setLightProperties(index, const LightProperties&), fog, environment ambient color.
• Resources: newRenderTexture(Pixmap*, flags), index and vertex buffers.
• Drawing: drawTriangles (an index buffer and four vertex buffers), drawLine, drawPointSprites, drawTrianglesFromMemory (vertex attribute arrays).
• Two textures per draw (setTextures(texture0, texture1)).

SceneWindow.dll defines the base class: its constructor and destructor, defaults for the buffer methods and drawTriangles (IMVU's OpenGL plugin uses them), and an asserting enable2dFrameBuffer. Its fields, after the vtable pointer: a magic number (0x3b913159, checked by the destructor) and a size counter (12 bytes in all). Outside the virtual methods, IMVU's code only uses an engine through helpers that call its virtual methods (render states' apply, resources' allocateInternalResource).

ABI

• 32-bit; virtual methods are __thiscall. Modern MSVC and clang-cl lay out single-inheritance vtables and pass arguments the same way as Visual C++ 2010, which built the client.
• boost::shared_ptr (Boost 1.61): the object pointer, then the control block: a vtable (destructor, dispose, destroy, get_deleter, get_untyped_deleter), the use count and the weak count, changed atomically. include/t5de/imvu/shared_ptr.hpp matches it. With non-trivial copy and destruction, it's returned through a hidden pointer and passed on the stack, as IMVU's code expects.
• Still to work out, for a renderer of our own: Mat4f, Transform, LightProperties, TextureTransform, Pixmap (pixmap.dll), the buffers' contents, VertexAttributeArray<T>, std::vector<PointSprite> as Visual C++ 2010 lays it out, and RenderBlendMode's values.
• Memory crosses the boundary only through virtual methods (destructors, control blocks), so each side can use its own C runtime. Exceptions must not cross it.

What a renderer can do

• Every draw passes through the interface between beginRendering and endRendering, so a renderer can record a frame and draw it however it likes: shaders emulating the fixed-function state, per-pixel lighting, anti-aliasing, anisotropic filtering, correct gamma and tone mapping, shadows and screen-space effects from the recorded frame, post-processing.
• It receives only two textures, a few lights, fog and blend state per draw. Newer material features (normal, roughness, emissive maps) aren't passed: they'd need hooks where SceneWindow.dll loads products, and the maps must exist in the product data the servers send Classic (unverified).
• Frame rate is not the renderer's: the scene advances in 33 ms ticks, and animations one frame per tick (ActionInstance::step calls incrementActionFrame per tick; FrameIntervalMaker::getFrameInterval counts the ticks; runApp waits out zero.dll's millisecondsPerFrame). Shorter ticks speed animations up. Smooth high frame rates need in-between poses: hooks in SceneWindow.dll's animation update.

What the client draws

From the pass-through (Direct3D, client 554.8, a few rooms):

• A room frame: about 146 drawTriangles, each with its own setModelSpace (131) and setTextures (138), with per-object alpha test, culling, normal inverting and blending, 252 texture-transform resets, 8 light updates; no drawLine, drawPointSprites or drawTrianglesFromMemory in those rooms.
• Every frame creates index and vertex buffers (4 and 12 in a room, with nothing new loading): most likely avatars, skinned on the CPU and uploaded again each frame.
• Each 3D window, and each room change, creates a new engine (createRenderEngine), so engines start and stop often; frame buffers are resized freely (up to 3440x1272 here).
• IMVU's OpenGL engine needs a real graphics driver (ARB_multitexture); on Windows' basic display driver (virtual machines) it throws and the client falls back. Its exceptions are std::exceptions our code can catch.

Plan

1. Pass-through plugin (done, verified in rooms): wraps IMVU's Direct3D engine, made with SceneWindow.dll's exported createRenderer, and logs what each frame calls, proving our declarations.
2. Work out the layouts above.
3. A renderer reproducing the current look on Direct3D 11 (it also runs in software, on WARP, where OpenGL can't), then the improvements.