Designing a Scalable Gamification Engine — Part 2 — Data Schema

This is the second in a series of entries detailing my journey to create a robust, scalable, and performant platform that enables gamification for other applications.

Building upon my previous article, we are in the midst of an attempt to design a gamification engine that has the following requirements:

1. Allow client applications to define goals for certain entities that are completed when those entities fulfill a goal’s criteria.
2. Allow clients to feed usage information into our system so that it tracks entity progress towards completing those goal requirements.
3. Allow clients to query for an entity’s progress towards its goals.

The proposed architecture is as follows:

Proposed system architecture

Defining our Goal Schema

It is my opinion that events should be this system’s basis for defining goals and updating progress towards goals. So, our schema design should facilitate goal definition such that goal requirements can be associated with the events sent as users interact with their applications.

Here is a rough example of how a goal could be expressed in terms of system events. We establish a Power User badge (aka goal) that a given user earns after 5 login events are detected for their userId.

// Goal Definition
{
  id: "goal-1234",
  name: "Power User",
  desc: "Log in at least 5 times",
  targetEntityId: "userId",
  criteria: [
    {
      id: "criterion-9999",
      qualifyingEvent: {
        action: "log-in"
      },
      aggregation: "count",
      threshold: 5
    }
  ]
}

Some of these fields are self-explanatory, so I will cover just the important ones:

1. targetEntityId indicates which event field our engine should track progress towards as events are received. In our example, we want to track progress towards a user logging in 5 times, so we will count login events for each individual userId that shows up on incoming system events.
2. criteria is an array of requirements that must each be individually fulfilled for an entity to have completed their overall goal. In our example, we have exactly one criteria: detect a ‘login’ event at least 5 times for the user. Once this criterion is met, the goal will be fulfilled for that user.
3. criteria.[x].qualifyingEvents describes the event metadata that must be present for an event to qualify towards a criteria. In our example, we will consider any event with action=log-into qualify as progress towards this goal for a given user.
4. criteria.[x].aggregation and criteria.[x].threshold describe how the events should occur for our criteria to be satisfied. In our case, we require a count of at least 5 log-in events to occur for the criteria to be satisfied.

The schema offers a some points of flexibility:

1. Anything can have a goal, not just system users. You can assign goals to any entity with a unique identifier (as long as the identifier is bundled with corresponding system events). Real world example: a user could get a badge for logging in 5 times, and his team of friends could also get an award for logging in 25 times between the group.
2. Criteria being an array allows for goals to be satisfied by events that take completely different forms. This allows “journeys” to be created where a user fulfills a goal by inducing many different (and disparate) system events. Real world example: a user could get a “Never Give Up” badge for failing a quiz at least once and then passing a quiz at least once.

The salient attribute of this schema, however, is that the qualifying event definition is intentionally generic — it is just a map of user-defined keys and values. To avoid coupling with clients and to promote broad usage, our gamification engine cannot make assumptions about which fields will be present within rules or events. A client should be able to define a goal with event data which is relevant to its application.

Defining our Event Schema

Next, we want to define the schema that our system will accept for events relevant to goal progress. Again, these events are coming in from external systems and we want those systems to know as little about our gamification engine as possible (as to prevent coupling). So, ultimately, there is no true required schema since the goal definition dictates what field values we are looking for.

Here is a simple event which our system could receive which would allow ‘john-doe-1234’ to make some progress towards the sample goal.

// Event that contributes towards achieving the goal
{
  action: "log-in",
  userId: "john-doe-1234"
}

Defining the Entity-Progress Schema

Lastly, we also want to define a schema that describes progress that entities make towards their goals as relevant events come in. Given our goal schema is comprised of multiplecriteria, we will need to track each individual’s progress towards each individual criterion. The following should do the trick:

// Sample entity progress towards goals
{
  id: "john-doe-1234",
  goals: {
    goal-1234: {
      isComplete: false,
      criteria: {
        criterion-9999: {
          isComplete: false,
          value: 1
        }
      }
    }
  }
}

This schema provides a singular record describing the entity’s progress towards all of its goals and towards the individual criteria within those goals. Our engine will be responsible for updating this record as relevant events are processed.

We will stop here for today. Our next task will be to define an approach for reliably and quickly processing system events that could contribute towards a goal’s progress.