Skip to content

Monitor an MMO

Full Recipe

Shared by: Ethan Bell

Model real-time player kill data from Planetside 2. Use API calls to supplement the killfeed graph with detailed information about the player characters and the weapons used.

Monitor an MMO Recipe 
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
version: 2
title: Planetside 2
contributor: https://github.com/emanb29
summary: Models real-time player kill data from Planetside 2 and supplements the killfeed graph with detailed information about the player characters and the weapons used.
description: |-
  Ingests the websockets killfeed from Daybreak Games' MMOFPS "PlanetSide 2", invoking the getJsonLines procedure to lazily fill out unknown static data. Replace all instances of `s:example` with a service-id acquired from http://census.daybreakgames.com/#service-id

ingestStreams:
  - name: planetside-killfeed
    source:
      type: WebsocketClient
      url: wss://push.planetside2.com/streaming?environment=ps2&service-id=s:example
      format:
        type: Json
      initMessages:
        # A couple notes: character names are not reused across servers, so we can subscribe to all servers ("worlds") and not worry about namespacing character names
        # Characters can be *renamed*, but this is rare because it costs the player $25
        - |-
          {
            "service":"event",
            "action":"subscribe",
            "worlds": ["all"],
            "characters":["all"],
            "eventNames":["Death"]
          }
      characterEncoding: UTF-8
    query: |-
      WITH * WHERE $that.type = 'serviceMessage'
      CREATE (m:murder) // these are never replayed, so no reason to idFrom
      SET m = COALESCE($that.payload, {})
      WITH id(m) as mId
      MATCH (murder) WHERE id(murder) = mId
      MATCH (victim) WHERE id(victim) = idFrom('character', murder.character_id)
      MATCH (attacker) WHERE id(attacker) = idFrom('character', murder.attacker_character_id)
      MATCH (weapon) WHERE id(weapon) = idFrom('weapon', murder.attacker_weapon_id)
      SET weapon.uninitialized = weapon.weapon_id IS NULL // flag the weapon for initialization if applicable
      SET victim:character, attacker:character, weapon:weapon,
        victim.character_id = murder.character_id, attacker.character_id = murder.attacker_character_id,
        weapon.weapon_id = murder.attacker_weapon_id
      CREATE (victim)<-[:victim]-(murder)-[:attacker]->(attacker), (murder)-[:weapon]->(weapon)
      // characters contain mutable data, eg certs. We'll add the timestamp to give us something to hook for refreshing data
      WITH murder, victim, attacker
      UNWIND [victim, attacker] AS character
      SET character.last_update = murder.timestamp

standingQueries:
  # Populate character data
  - name: populate-character-data
    pattern:
      type: Cypher
      # match each new character-label node
      query: MATCH (newCharacter:character) WHERE newCharacter.character_id IS NOT NULL RETURN DISTINCT id(newCharacter) AS id
      mode: DISTINCT_ID
    outputs:
      - name: populate-fresh-character
        preEnrichmentTransformation:
          type: InlineData
        resultEnrichment:
          query: |-
            MATCH (c)
            WHERE id(c) = $that.id
            CALL loadJsonLines("https://census.daybreakgames.com/s:example/get/ps2:v2/character/?character_id="+c.character_id) YIELD value
            SET c += COALESCE(value.character_list[0], {}) // there should always be a "character_list" with exactly 1 value: the character we queried
            RETURN null
          parameter: that
        destinations:
          - type: Drop
  # Populate weapon data
  - name: populate-weapon-data
    pattern:
      type: Cypher
      query: MATCH (weapon:weapon) WHERE weapon.uninitialized = true AND weapon.weapon_id IS NOT NULL RETURN DISTINCT id(weapon) AS id
      mode: DISTINCT_ID
    outputs:
      - name: populate-weapon
        preEnrichmentTransformation:
          type: InlineData
        resultEnrichment:
          query: |-
            MATCH (weapon) WHERE id(weapon) = $that.id
            CALL loadJsonLines("https://census.daybreakgames.com/s:example/get/ps2:v2/item?item_id="+weapon.weapon_id+"&c:join=weapon_datasheet") YIELD value
            SET weapon += COALESCE(value.item_list[0], {}) // there should always be a "item_list" with exactly 1 value: the weapon we queried
            REMOVE weapon.uninitialized
            RETURN null
          parameter: that
        destinations:
          - type: Drop
  # Future Standing Query idea: monitor for "trades" (ie, when two players kill each other simultaneously)

nodeAppearances:
  - predicate:
      propertyKeys: []
      dbLabel: character
      knownValues: {}
    icon: ion-android-person
  - predicate:
      propertyKeys: []
      dbLabel: murder
      knownValues: {}
    icon: "\u2694\uFE0F"
  - predicate:
      propertyKeys: []
      dbLabel: weapon
      knownValues: {}
    icon: "\uD83D\uDD2B"
  - predicate:
      propertyKeys: []
      knownValues: {}

quickQueries: []
sampleQueries: []

Download Recipe

Scenario

Ingest the killfeed websocket output from Daybreak Games' MMOFPS "PlanetSide 2", invoking the getJsonLines procedure to lazily fill out unknown static data. Replace all instances of s:example with a service-id acquired from http://census.daybreakgames.com/#service-id

Sample Data

Death events will stream in from the websocket once the recipe is started. The events are JSON objects like the one below.

{
    "payload":{
        "attacker_character_id":"5428010618015189713",
        "attacker_fire_mode_id":"26103",
        "attacker_loadout_id":"15",
        "attacker_vehicle_id":"0",
        "attacker_weapon_id":"26003",
        "character_id":"5428168624838258657",
        "character_loadout_id":"6",
        "event_name":"Death",
        "is_headshot":"1",
        "timestamp":"1392056954",
        "vehicle_id":"0",
        "world_id":"1",
        "zone_id":"2"
    },
    "service":"event",
    "type":"serviceMessage"
}

How it Works

The recipe connects an ingest stream to the PS2 Event Streaming WebSocket to manifest Death events as murder, victim, attacker, weapon, and character nodes in Quine.

The ingest stream processes events emitted from the WebSocket:

ingestStreams:
  - name: planetside-killfeed
    source:
      type: WebsocketClient
      url: wss://push.planetside2.com/streaming?environment=ps2&service-id=s:example
      format:
        type: Json
      initMessages:
        - |-
          {
            "service":"event",
            "action":"subscribe",
            "worlds": ["all"],
            "characters":["all"],
            "eventNames":["Death"]
          }
      characterEncoding: UTF-8
    query: |-
      WITH * WHERE $that.type = 'serviceMessage'
      CREATE (m:murder)
      SET m = COALESCE($that.payload, {})
      WITH id(m) as mId
      MATCH (murder) WHERE id(murder) = mId
      MATCH (victim) WHERE id(victim) = idFrom('character', murder.character_id)
      MATCH (attacker) WHERE id(attacker) = idFrom('character', murder.attacker_character_id)
      MATCH (weapon) WHERE id(weapon) = idFrom('weapon', murder.attacker_weapon_id)
      SET weapon.uninitialized = weapon.weapon_id IS NULL
      SET victim:character, attacker:character, weapon:weapon,
        victim.character_id = murder.character_id, attacker.character_id = murder.attacker_character_id,
        weapon.weapon_id = murder.attacker_weapon_id
      CREATE (victim)<-[:victim]-(murder)-[:attacker]->(attacker), (murder)-[:weapon]->(weapon)
      WITH murder, victim, attacker
      UNWIND [victim, attacker] AS character
      SET character.last_update = murder.timestamp
POST /api/v2/graph/quine/ingests
{
  "name": "planetside-killfeed",
  "source": {
    "type": "WebsocketClient",
    "url": "wss://push.planetside2.com/streaming?environment=ps2&service-id=s:example",
    "format": {
      "type": "Json"
    },
    "initMessages": [
      "{\n  \"service\":\"event\",\n  \"action\":\"subscribe\",\n  \"worlds\": [\"all\"],\n  \"characters\":[\"all\"],\n  \"eventNames\":[\"Death\"]\n}"
    ],
    "characterEncoding": "UTF-8"
  },
  "query": "WITH * WHERE $that.type = 'serviceMessage'\nCREATE (m:murder)\nSET m = COALESCE($that.payload, {})\nWITH id(m) as mId\nMATCH (murder) WHERE id(murder) = mId\nMATCH (victim) WHERE id(victim) = idFrom('character', murder.character_id)\nMATCH (attacker) WHERE id(attacker) = idFrom('character', murder.attacker_character_id)\nMATCH (weapon) WHERE id(weapon) = idFrom('weapon', murder.attacker_weapon_id)\nSET weapon.uninitialized = weapon.weapon_id IS NULL\nSET victim:character, attacker:character, weapon:weapon,\n  victim.character_id = murder.character_id, attacker.character_id = murder.attacker_character_id,\n  weapon.weapon_id = murder.attacker_weapon_id\nCREATE (victim)<-[:victim]-(murder)-[:attacker]->(attacker), (murder)-[:weapon]->(weapon)\nWITH murder, victim, attacker\nUNWIND [victim, attacker] AS character\nSET character.last_update = murder.timestamp"
}

A standing query matches each new character node to populate the character properties via the census API.

standingQueries:
  - name: populate-character-data
    pattern:
      type: Cypher
      query: MATCH (newCharacter:character) WHERE newCharacter.character_id IS NOT NULL RETURN DISTINCT id(newCharacter) AS id
      mode: DISTINCT_ID
    outputs:
      - name: populate-fresh-character
        preEnrichmentTransformation:
          type: InlineData
        resultEnrichment:
          query: |-
            MATCH (c)
            WHERE id(c) = $that.id
            CALL loadJsonLines("https://census.daybreakgames.com/s:example/get/ps2:v2/character/?character_id="+c.character_id) YIELD value
            SET c += COALESCE(value.character_list[0], {})
            RETURN null
          parameter: that
        destinations:
          - type: Drop
POST /api/v2/graph/quine/standingQueries
{
  "name": "populate-character-data",
  "pattern": {
    "type": "Cypher",
    "query": "MATCH (newCharacter:character) WHERE newCharacter.character_id IS NOT NULL RETURN DISTINCT id(newCharacter) AS id",
    "mode": "DISTINCT_ID"
  },
  "outputs": [
    {
      "name": "populate-fresh-character",
      "preEnrichmentTransformation": {
        "type": "InlineData"
      },
      "resultEnrichment": {
        "query": "MATCH (c)\nWHERE id(c) = $that.id\nCALL loadJsonLines(\"https://census.daybreakgames.com/s:example/get/ps2:v2/character/?character_id=\"+c.character_id) YIELD value\nSET c += COALESCE(value.character_list[0], {})\nRETURN null",
        "parameter": "that"
      },
      "destinations": [
        {
          "type": "Drop"
        }
      ]
    }
  ]
}

The character_list object returned from the API is similar to the one below. 

{"character_list":[{"character_id":"5428010618020694593","name":{"first":"Dreadnaut","first_lower":"dreadnaut"},"faction_id":"1","head_id":"1","title_id":"97","times":{"creation":"1353434436","creation_date":"2012-11-20 18:00:36.0","last_save":"1508990907","last_save_date":"2017-10-26 04:08:27.0","last_login":"1508985950","last_login_date":"2017-10-26 02:45:50.0","login_count":"971","minutes_played":"101148"},"certs":{"earned_points":"206533","gifted_points":"9358","spent_points":"205791","available_points":"10100","percent_to_next":"0.002999999962901"},"battle_rank":{"percent_to_next":"43","value":"102"},"profile_id":"21","daily_ribbon":{"count":"5","time":"1508911200","date":"2017-10-25 06:00:00.0"},"prestige_level":"0"}],"returned":1}

A second standing query matches each new weapon node to populate the weapon properties via the census API.

standingQueries:
  - name: populate-weapon-data
    pattern:
      type: Cypher
      query: MATCH (weapon:weapon) WHERE weapon.uninitialized = true AND weapon.weapon_id IS NOT NULL RETURN DISTINCT id(weapon) AS id
      mode: DISTINCT_ID
    outputs:
      - name: populate-weapon
        preEnrichmentTransformation:
          type: InlineData
        resultEnrichment:
          query: |-
            MATCH (weapon) WHERE id(weapon) = $that.id
            CALL loadJsonLines("https://census.daybreakgames.com/s:example/get/ps2:v2/item?item_id="+weapon.weapon_id+"&c:join=weapon_datasheet") YIELD value
            SET weapon += COALESCE(value.item_list[0], {})
            REMOVE weapon.uninitialized
            RETURN null
          parameter: that
        destinations:
          - type: Drop
POST /api/v2/graph/quine/standingQueries
{
  "name": "populate-weapon-data",
  "pattern": {
    "type": "Cypher",
    "query": "MATCH (weapon:weapon) WHERE weapon.uninitialized = true AND weapon.weapon_id IS NOT NULL RETURN DISTINCT id(weapon) AS id",
    "mode": "DISTINCT_ID"
  },
  "outputs": [
    {
      "name": "populate-weapon",
      "preEnrichmentTransformation": {
        "type": "InlineData"
      },
      "resultEnrichment": {
        "query": "MATCH (weapon) WHERE id(weapon) = $that.id\nCALL loadJsonLines(\"https://census.daybreakgames.com/s:example/get/ps2:v2/item?item_id=\"+weapon.weapon_id+\"&c:join=weapon_datasheet\") YIELD value\nSET weapon += COALESCE(value.item_list[0], {})\nREMOVE weapon.uninitialized\nRETURN null",
        "parameter": "that"
      },
      "destinations": [
        {
          "type": "Drop"
        }
      ]
    }
  ]
}

The item_list object returned from the API is similar to the one below.

{"item_list":[{"item_id":"7","item_type_id":"26","item_category_id":"139","is_vehicle_weapon":"0","name":{"de":"Spawn -Leuchte","en":"Spawn Beacon","es":"Baliza de apariciones","fr":"Balise de réapparition","it":"Faro di rigenerazione","tr":"Animation Control Sign"},"description":{"de":"Eine Signalleuchte zum schnellen Absetzen von Truppmitgliedern in einen Bereich aus geringer Höhe.","en":"A signal-emitting beacon that allows squad members to drop pod into an area from low orbit.","es":"Una baliza que emite una señal y permite a los miembros del escuadrón aterrizar con cápsula dentro de una zona desde una órbita baja.","fr":"Une balise émettrice de signaux qui module de aux membres de l'es un déploiement à partir d'une orbite basse.","it":"Faro emettitore di segnali che permette ai membri della squadra di inserirsi a caldo nell'area.","tr":"It allows squad members to come from low orbit to instantaneous space is a scattering checkmark."},"faction_id":"0","max_stack_size":"1","image_set_id":"1568","image_id":"3056","image_path":"/files/ps2 /images/static/3056.png","is_default_attachment":"0"}],"returned":1}

Running the Recipe

Warning

Before running this recipe beyond a few seconds, you'll need to apply for a service ID from Daybreak Games in order to access their API. The service ID "s:example" is available for casual use--it is throttled to 10 requests per minute per client IP address.

Update a local copy of this recipe once you receive your personal service-id.

Please don't share the updated recipe or your service ID with others.

❯ java -jar quine-2.2.1.jar -r planetside-2.yaml
Graph is ready
Using 4 node appearances
Running Standing Query populate-character-data
Running Standing Query populate-weapon-data
Ingest Stream planetside-killfeed: Could not verify that the provided ingest query is idempotent. If timeouts occur, query execution may be retried and duplicate data may be created.
Running Ingest Stream planetside-killfeed
Quine web server available at http://127.0.0.1:8080
Default API version: v2

Once the recipe is running, you can explore the graph to find subgraphs formed around Death events.

Murder Event