Simplified JSON notation

JSON is human-editable, but not necessarily human-friendly. A typical JSON configuration file:

{
    "ip" : "127.0.0.1",
    "port" : 666
}

A more Lua-inspired syntax is friendlier:

ip = "127.0.0.1"
port = 666

This syntax corresponds 1-1 with regular JSON syntax and can be trivially converted back and forth with the following rules:

• Assume an object definition at the root level (no need to surround entire file with { } ).
• Commas are optional
• Quotes around object keys are optional if the keys are valid identifiers
• Replace : with =

On the other hand, all syntax wars are pointless and will only send us into an early grave.

Multithreaded gameplay

How do we multithread gameplay without driving gameplay programmers insane?

My current idea is:

• Do all gameplay processing as events reacting to stuff (such as collide_with_pickup_object), not through a generic update() call.
• Each event concerns a number of entities (e.g., [player, ammo_pack]). The processing function for an event is allowed to touch the entities it concerns freely, but not any other entities.
• Each frame, consider all events. Let two entities being in the same event define an equivalence relation between those two entities. The corresponding equivalence classes then define "islands" of entities that can be processed safely on separate cores.
• Assign each island to a core, process the events for that island one by one on the core.
• Provide a thread-safe interface to any global entitites that the event processors may need to touch for effect spawning, sound play, etc. (Preferrably through a queue so that the global entities don't have to be touched directly from the event processors.)

Some concerns:

• Will the islands become "too big". I.e., if almost everything interacts with the player, there is a risk that everything ends up in a single big "player island".
• Will it be reasonable for gameplay programmers to write code that follows these restrictions.

Two way serialization function

A trick to avoid having to keep the serialization code for input and output in sync is to use the same code for both input and output:

struct Object {

template <>

STREAM & serialize(STREAM & stream) {

return stream & a & b & c;

}

int a, b, c;

};

Here we have used & as our serialization operator. We could use any operator we like.

We then just implement the operator to do the right thing for our input and output streams:

template < > InputArchive & operator &(InputArchive &a, int &v) {

a.read(&v, sizeof(v));

return a;

}




template < > OutputArchive & operator & (OutputArchive &a, int &v) {

a.write(&v, sizeof(v));

return a;

}

These are both template specializations of a generic streaming template.

template <>

STREAM & operator &(STREAM & stream, T & t) {

t.serialize(stream);

}

Now we can stream all kinds of types either by implementing serialize in the type or by defining a template specialization of operator & for that type.

Simple perfect murmur hashing

A simple way of finding a perfect (collision free) murmur hash for a set of keys S is to simply iterate over the seed values until we find one that doesn't produce any collisions:

seed := 0
while true
    H[i] := murmur_hash(S[i], seed) for all i
    return seed if no_duplicates(H)
    seed := seed + 1

As long as the size of the key set S is not much bigger than the square root of the output range of the hash function, the algorithm above will terminate quickly. For example, for a 32 bit hash this algorithm works well for sets up to about 65 000 elements. (In fact we can go up to 100 000 elements and still find a good seed by just making a couple of extra iterations.)

With a perfect hash function we only need to compare the hash values to dermine if two keys are equal, we never have to compare (or even store) the original keys themselves. We just have to store the 32-bit seed and the hash values. This saves both memory and processing time.

In the BitSquid engine this simple perfect hashing scheme is used to generate 32-bit resource IDs from resource names and types.

JSON configuration data

The BitSquid engine will use JSON as an intermediate format for all generic configuration data.

JSON is better than a custom binary format because:

• The data can be inspected and debugged manually.
• There are lots of editors.
• Changes merge nicer in SVN.
• The data is platform independent.
• As long as you are just adding data fields, the data is both backward and forward compatible.

JSON files are slower to parse than binary files, but that doesn't matter because it is only an intermediate format. They are bigger, but not that much bigger, and again it doesn't matter because it is only an intermediate format. We will generate efficient binary data for the runtime.

JSON is better than XML because:

• It is a lot simpler and easier to parse.
• It maps directly to native data structures.
• It is typed, meaning you can understand (more of) it without needing a DTD.
• It is more "normalized". (In XML you have to choose whether to put information in attributes or in text nodes.

XML is good for marking up text, but not so good for describing data.

Welcome to the BitSquid blog

This blog will collect rants, ideas and random thoughts about the development of the BitSquid game engine.

See: http://www.bitsquid.se for more information.