Friday, October 23, 2009

Picking a scripting language

We are planning to make the BitSquid engine largely scripting language agnostic. We will expose a generic scripting interface from the engine and it should be relatively easy to bind that to whatever scripting language you desire.

Still, we have to pick some language to use for our own internal projects and recommend to others. I'm currently considering three candidates:


  • Use regular C/C++ for scripting.
  • Run it dynamically either by recompiling and relinking DLLs or by running an x86 interpreter in the game engine and loading compiled libs directly.
  • + Static typing
  • + Syntax checking & compiling can be done with an ordinary compiler
  • + When releasing the game we can compile to machine code and get full native speed
  • - C is not that nice for scripting
  • - Huge performance differences between "fully compiled" and "interactive" code makes it difficult for the gameplay programmers to do performance estimates.
  • Lua has the same feature set as Python and Ruby, but is smaller, more elegant and faster.
  • Other scripting langues such as Squirrel, AngelScript offer reference counting and static typing, but are not as well known / used
  • + Dynamic, elegant, small
  • + Something of a standard as a game scripting language
  • + LuaJIT is very fast
  • - Non-native objects are forced to live on the heap
  • - Garbage collection can be costly for a realtime app
  • - Speed can be an issue compared to native code
  • - Cannot use LuaJIT on consoles
  • Use the Mono runtime and write scripts in C#, Boo, etc.
  • + Static typing
  • + Popular, fast
  • - Huge, scary runtime
  • - Garbage collection
  • - Requires license to run on console
  • - Can probably not JIT on console


  1. Lua all the way! We're about to ship a game using it on Xbox 360; by using it I mean 95% of the game code, including UI, is written entirely in Lua.

    We had some garbage collection issues - we had to add a touch of "manual garbage collection" here and there, which took a day or two of the final stretch - but that was all that it took memory-wise, after months and months of a room full of gameplay programmers banging away at code blissfully oblivious of the existence of such a thing as "memory". The speed of development is orders of magnitude higher than C++, and the code is considerably smaller.

    Performance is the main issue - we dedicated a full Xenon core to it, and got away with it - but on the PS3 that won't be possible. You either have to design around it - avoid game designs requiring lots of logic - or go to C++, where you lose the productivity gain.

    Our game is VERY simulation-heavy, IMHO higher than the vast majority of games on the market - so it was kind of a worst case; on the other hand, I understand how you, thinking only about an "engine", wouldn't want to limit potential customers. But the limit is very high - and the alternative is C++.

    What's especially great about Lua is the native support for coroutines, which allow you to implement what's called "latent functions" in UnrealScript - which make game logic code much more straightforward than the alternatives (e.g. state machines).

  2. You can also make your own scripting language that is job-centric and state-machine aware.

  3. Yes, I'm leaning towards Lua myself. It is what we used in the Grin engine and overall it did great things for us.

    The main problem we had with Lua at Grin was performance, especially in Terminator Salvation. We didn't run Lua on a separate core, and since we were shipping for PS3, that wouldn't really have helped. I think the performance problems might be a result of trying to do "too much" in Lua and that it might be mediated by moving more of the heavy-duty stuff to C++ and being a bit smarter about the C/Lua interface. But it is scary to end up in that situation late in a project. On the other hand, I guess you always end up with performance problems in gameplay code, no matter what the situation is.

    What is slightly scary is that when optimizing C++ code it usually boils down to a few bottleneck operations. You rewrite them and get a quick performance boost. With Lua, there weren't any real such choke points. It was just a matter of a slow (compared to C, not to other scripting languages) virtual machine running lots and lots of code, so there was no single place to do quick and easy optimizations.

    Memory might also be an issue, since we want a separate lua state per core (to allow fully multithreaded gameplay), that will multiply the lua runtime size by the number of cores. But I think we can solve that by keeping all the state data in C and just doing operations on it in Lua. (That also saves the trouble of synchronizing the Lua states.)

  4. At Unity we have very good experience with Mono. It gives us excelent programming languages and very good performance over interpreted languages.
    So if you get over the admittedly high startup costs, in terms of integration and understanding the code, there are many benefits.
    Also we get very good support from the Novell guys.

    You are velcome to contact us if yo need more info.

  5. Mix it C++ + javascript ..