LuaJIT Internals: Intro

Welcome to the LuaJIT blog series. In this one, we will go over some basic introduction, stuff like the motivation behind this research, setup notes, etc.

Table of Contents

• LuaJIT Internals(Pt. 0/3): Intro / This post
• LuaJIT Internals(Pt. 1): Stepping into the VM
• LuaJIT Internals(Pt. 2): Fighting the JIT Compiler
• LuaJIT Internals(Pt. 3): Crafting Shellcodes
• (Bonus!) LuaJIT Sandbox Escape: The Sage Ends

Motivation

Some of the reasons behind this work are:

• Pure curiousity, I love to dig into interpreters, and to understand the inner workings of a programming language.
• When scrolling through the luajit wiki docs, I found that some topics are partially documented(they start to describe something and put in the middle of a paragraph sentences like “TODO: should elaborate more on…”, etc.).
• Figuring out the inner workings of LuaJIT often requires you to read the sources and “reverse-engineer” your way up. These posts aims to connect the missing dots between the documentation and the source code(+add relevant snippets).

Setup

The LuaJIT version & comiple flags I used for this research:

make -j$(nproc) CCDEBUG=-g3 XCFLAGS=-DLUAJIT_USE_GDBJIT

(Release: v2.1.0-beta3, 8271c64)

XCFLAGS will be useful for JIT debugging, and the CCDEBUG adds debugging more information so it will be easier to debug the C sources.

The TValue data structure

Before we begin talking about the internals of LuaJITL: It’s important to know a very basic data structure, and that’s the TValue:

In LuaJIT, every variable is represented using a TValue(tagged-value). Every TValue is 64bit wide and has two parts:

• If the variable has a type of Number: The value of the variable is ‘embedded’ into the TValue itself and will use all of the 64bit. The number will be saved as a double.
• If the variable has a more ‘complex’ type than a Number(like, string, table, function, etc.):
  • The first 32bit contains type information about the variable
  • The second 32bit contains a pointer to the object

Visually, this is how the stack of the LuaJIT VM looks like:

Further reading

You won’t need too much knowlege about the internals of Lua in order to read this blog series. However, you do need a *basic* understanding about some Lua topics(concepts like the lua_State struct and general idea about the Lua stack), I’d suggest reading this and this for a start.

I hope you’ll learn something new, good luck!