ECS in JS – Storage Mechanisms

I maintain a JavaScript library called Javelin: a toolkit for building multiplayer games in TypeScript. At the core of Javelin is an Entity-Component System (ECS). In this article, I discuss ECS storage mechanisms, Javelin's current strategy, and how it might be improved in the near future.

Storage and Iteration

ECS is all about iteration. ECS programs store game data (components) in iterable collections. Game logic is written inside of loops which iterate these collections.

A Javelin component looks like:

{ x: 0, y: 0 }

Look familiar? This paradigm has several advantages. Game state can be realized with native syntax that supports rich, complex types. Javelin is highly compatible with third-party libraries because most NPM packages also use objects. Objects also follow the semantics found in in decades of literature, articles, and forum discussions, making programs which use them accessible to a wider audience.

However, there are some great alternatives that have become more popular in JavaScript ECS libraries which Javelin currently doesn't support. We'll discuss those later, but first, let's look at how Javelin stores component data.

Arrays of objects

Javelin uses arrays of objects to store game data. Each array holds a single kind of object, each with the same structure.

These arrays look something like the following:

const table = [
  [{ x: 0, y: 0 }, ...], // position
  [{ x: 1, y: 1.5 }, ...], // velocity
]

This multi-dimensional array (or table) is called an archetype. Each column of the table is an array of like objects, often referred to as an array-of-structs (AoS). A game entity is a pointer to a single "row" in the table. Entities in archetypal ECS are fully described by this row (their components), and don't have methods of their own.

Iterating and updating entity data in Javelin is intuitive:

const [p, v] = table
for (let i = 0; i < entities.length; i++) {
  p[i].x += v[i].x
  p[i].y += v[i].y
}

There are some downsides to the array-of-structs approach. Accessing and updating object members is slower than array indices. Objects must be copied when shared with a worker thread, and also is slower to serialize (e.g. to a JSON string or ArrayBuffer) when compared to raw binary data.

Arrays of numbers

Arrays of numbers are faster to iterate and update. Moving from the above object example to arrays of numbers would look something like:

[
  {
    x: [0],
    y: [0],
  },
  {
    x: [1],
    y: [1.5],
  },
]

Game state is stored in a struct of arrays (SoA) with this approach. The syntax of iterating and updating values within SoA is less intuitive than AoS, but may be faster because arrays can be better optimized by the CPU in a type of computation called vectorization.

Iteration looks a little different with SoA. Pay attention to how values are accessed in the following example:

const [p, v] = table
for (let i = 0; i < entities.length; i++) {
  p.x[i] += v.x[i]
  p.y[i] += v.y[i]
}

Not too bad. Since each individual component property is now backed by its own array, we have to access them a bit differently.

The range of values you can store in SoA is limited. Sets, Maps and other complex types are generally unsupported. SoA also bars the archetype from storing a third-party library object in an archetype column.

TypedArrays

We can go even further!

TypedArrays can be rapidly serialized an de-serialized (e.g. in a network message), and shared with worker threads without wasteful data copying. Several ECS libraries are built entirely around the TypedArray approach with some really great performance results. The TypedArray-based ECS BitECS has grown in popularity and is even being adopted into larger game libraries like Phaser 4.

But TypedArrays are an advanced language feature. Not everyone with the ambitions to make a game want or will care to use them. This has led to stop-gap solutions for library authors who wish to appeal to a wide audience, as is the case with BitECS, whose authors propose a strategy to hand-write proxy classes in order to masquerade TypedArray data as plain objects.

I think solutions like proxies aren't all that useful. It's not a big ask for performance-minded developers to learn and utilize the syntax of SoA for their proprietary game state. The real cost lies in the lack of interop with objects you don't manage, i.e. third-party libraries, as you can't easily store them in SoA.

Object-Array Harmony

I have been experimenting with adding TypedArray-based components to Javelin for those people who wish to take advantage of their unique benefits. It was pretty easy to get a simple POC built.

In the POC, component data can be represented within an archetype with one or more TypedArrays.

const Position = makeBinarySchema({ x: number, y: number })
const Velocity = makeBinarySchema({ x: number, y: number })
const Body = [Position, Velocity] as const
const bodies = makeQuery(Body)
for (const [entities, [p, v]] of bodies) {
  p.x[i] += v.x[i]
  p.y[i] += v.y[i]
}

Binary components are fast. And to nobody's surprise, about as fast as the other TypedArray ECS alternatives out there.

Archetypes can also contain a mixture of both object columns and TypedArray columns. Since archetypes in Javelin can store third-party objects, this makes it easy to mix proprietary, binary game state with library dependencies.

Below is a pseudo-code example of using a TypedArray-based component alongside a Three.js mesh object, all stored right in the same archetype.

const Position = makeBinarySchema({ x: number, y: number })
const Mesh = makeSchema({ position: { x: number, y: number } })
const Player = [Position, Mesh] as const
const players= makeQuery(Player)

// attach position and mesh to entity `5`
attach(5, Player, [, new Three.Mesh()])

for (const [entities, [p, m]] of players) {
  for (let i = 0; i < entities.length; i++) {
    const mesh = m[i]
    m.position.x = p.x[i]
    m.position.y = p.y[i]
    m.position.z = p.z[i]
  }
}

Using the hybrid approach, you can reap the benefits of both SoA and AoS storage types.

Below is a table that shows the median time of a system which iterates and updates components of 5 million entities across three different storage paradigms (Node v16.4.2).

As you can see, performance increases as object components are replaced with binary components.

Conclusion

In conclusion, each ECS storage type has it's benefits and drawbacks. Arrays-of-structs are more ergonomic and compatible with third-party libraries, but slower to iterate, read, and mutate. Structs-of-arrays are lightning fast but don't play well with third-party libraries, support a limited range of component shapes, and are syntactically displeasing to some.

I'm working on adding support for binary components to Javelin to give users more options, and there's still a lot of work that needs to be done. I've published a repo of a hypothetical ECS that supports archetypes and hybrid SoA/AoS storage that I'm using as a reference while I make these improvements to Javelin.

Thanks for reading!