Re-Implementing Live2D runtime in LÖVE

 (video above shows my implementation in action using LÖVE framework)

Live2D is a nice thing, the fluid character movement gives additional touch to the game which uses it. For my personal use, however it has annoying limitation: Lua is not officially supported, especially LÖVE. Well, there's 2 ways to overcome this.

Writing external Lua C module

This is probably the simplest way (but not that easy). Link with Live2D C++ runtime library files, add code which interact with Lua (and LÖVE), then you got Live2LOVE. This module is actually very fast, considering the Lua to C API overhead, and works by letting Live2D to do the model transformation and LÖVE to do the model rendering. Since it uses official Live2D runtime, it has these limitations:

• Must link with OpenGL. This is not a problem since LÖVE uses OpenGL already.
• VS2017 is not supported (you have to use Cubism 3 for that). However it supports down to VS2010, but LÖVE requires VS2013, so this is not really a problem unless you compile LÖVE to use VS2017 runtime.
• MinGW/Cygwin compilation is not supported. Not really a problem since compiling LÖVE in Windows using MinGW/Cygwin itself is not supported.
• Linux and macOS is not supported. This is the real problem. Not all people use Windows to run LÖVE.

So another idea that comes to my mind is:

Re-Implement Live2D Cubism 2 Runtime

In Lua, because why not. This is actually somewhat time-consuming process and took me more than 3 weeks to have model rendering working as intended. My additional goal for this is to have Live2LOVE-compatible interface too, so switching between implementation is simply changing the "require" module. From now on, I'll refer Live2D Runtime as Live2D Cubism 2 Runtime.

I start by downloading Live2D Runtime for WebGL (Javascript) and beautify the code (since it's in .min.js file). As expected, the function method names are obfuscated. So, I unpacked Android version of Live2D C++ Runtime and deobfuscate the method name by matching the arguments from Live2D Runtime C++ header files and with help of IDA pseudocode decompiler to compare the implementation with the Javascript ones. This whole process took 2 weeks.

Then I start by writing the Lua equivalent code based on Javascript Live2D Runtime. This is the easiest to do since Javascript is also dynamically typed. Carefully translating 0-based indexing code to 1-based. Then fixing bugs, writing Live2LOVE-compatible interface so I can use existing Live2D viewer code that using Live2LOVE, and testing.

After a week, I got model rendering, motion, and expression working. Using existing code from my LÖVE-based Live2D model viewer to use my own implementation instead of Live2LOVE. Then the next problem comes: it's 4x slower than Live2LOVE. Live2LOVE took 1ms to render Kasumi (casual summer clothes) while my implementation took 4ms to render same model. I already code the implementation carefully so that LuaJIT happily accepts my code and won't bail out to interpreter as possible.

I started optimization by using "Data" objects instead of plain table when updating the "Mesh" object for drawing. This cuts down the update time significantly from 4ms to 1.7ms so using table to update the "Mesh" object is always a bad idea. Someone in LÖVE Discord then says "try to use all FFI instead of plain table". At first I did not agree with him because I want to preserve compatibility with mobile, but then I decide to proceed by having falling back to tables in case FFI is not suitable (JIT is off, FFI support is not enabled, or using vanilla Lua). I swapped most types from plain table to FFI objects and I can get as low as 1.2ms, almost close to Live2LOVE 1ms.

Conclusion

Re-implementing Live2D Runtime is a nice experience. It gives me better overview when to start optimizing code instead of optimizing early and overview of how Live2D model transformation works. Apparently it can't beat C++ version of the official Live2D Runtime in terms of model updating, but I think it can beat it in terms of model rendering. I'm thinking of "Mesh" batching technique, which is basically: accumulate vertices to render then draw'em'all at once if flush requested. I'm currently satisfied with the current result, but I think I still can do better

 

... and hope to God it can success without problems.