Monitor an MMO

Full Recipe

Shared by: Ethan Bell

Model real-time player kill data from Planetside 2. Use API calls to supplement the killfeed graph with detailed information about the player characters and the weapons used.

Monitor an MMO Recipe

version: 1
title: Planetside 2
contributor: https://github.com/emanb29
summary: Models real-time player kill data from Planetside 2 and supplements the killfeed graph with detailed information about the player characters and the weapons used.
description: |-
  Ingests the websockets killfeed from Daybreak Games' MMOFPS "PlanetSide 2", invoking the getJsonLines procedure to lazily fill out unknown static data. Replace all instances of `s:example` with a service-id acquired from http://census.daybreakgames.com/#service-id
ingestStreams:
  - type: WebsocketSimpleStartupIngest
    url: wss://push.planetside2.com/streaming?environment=ps2&service-id=s:example
    initMessages:
    # A couple notes: character names are not reused across servers, so we can subscribe to all servers ("worlds") and not worry about namespacing character names
    # Characters can be *renamed*, but this is rare because it costs the player $25
    - |- 
      {
        "service":"event",
        "action":"subscribe",
        "worlds": ["all"],
        "characters":["all"],
        "eventNames":["Death"]
      }
    format:
      type: CypherJson
      query: |-
        WITH * WHERE $that.type = 'serviceMessage'
        CREATE (m:murder) // these are never replayed, so no reason to idFrom
        SET m = COALESCE($that.payload, {})
        WITH id(m) as mId
        MATCH (murder) WHERE id(murder) = mId
        MATCH (victim) WHERE id(victim) = idFrom('character', murder.character_id)
        MATCH (attacker) WHERE id(attacker) = idFrom('character', murder.attacker_character_id)
        MATCH (weapon) WHERE id(weapon) = idFrom('weapon', murder.attacker_weapon_id)
        SET weapon.uninitialized = weapon.weapon_id IS NULL // flag the weapon for initialization if applicable
        SET victim:character, attacker:character, weapon:weapon,
          victim.character_id = murder.character_id, attacker.character_id = murder.attacker_character_id,
          weapon.weapon_id = murder.attacker_weapon_id
        CREATE (victim)<-[:victim]-(murder)-[:attacker]->(attacker), (murder)-[:weapon]->(weapon)
        // characters contain mutable data, eg certs. We'll add the timestamp to give us something to hook for refreshing data
        WITH murder, victim, attacker
        UNWIND [victim, attacker] AS character
        SET character.last_update = murder.timestamp
standingQueries:
  # Populate character data
  - pattern:
      type: Cypher
      # match each new character-label node
      query: MATCH (newCharacter:character) WHERE newCharacter.character_id IS NOT NULL RETURN DISTINCT id(newCharacter) AS id
    outputs:
      populate-fresh-character:
        type: CypherQuery
        query: |-
          MATCH (c)
          WHERE id(c) = $that.data.id
          CALL loadJsonLines("https://census.daybreakgames.com/s:example/get/ps2:v2/character/?character_id="+c.character_id) YIELD value
          SET c += COALESCE(value.character_list[0], {}) // there should always be a "character_list" with exactly 1 value: the character we queried
  # Populate weapon data
  - pattern:
      type: Cypher
      query: MATCH (weapon:weapon) WHERE weapon.uninitialized = true AND weapon.weapon_id IS NOT NULL RETURN DISTINCT id(weapon) AS id
    outputs:
      populate-weapon:
        type: CypherQuery
        query: |-
          MATCH (weapon) WHERE id(weapon) = $that.data.id
          CALL loadJsonLines("https://census.daybreakgames.com/s:example/get/ps2:v2/item?item_id="+weapon.weapon_id+"&c:join=weapon_datasheet") YIELD value
          SET weapon += COALESCE(value.item_list[0], {}) // there should always be a "item_list" with exactly 1 value: the weapon we queried
          REMOVE weapon.uninitialized
  # Future Standing Query idea: monitor for "trades" (ie, when two players kill each other simultaneously)
nodeAppearances:
  - predicate:
      propertyKeys: []
      dbLabel: character
      knownValues: {}
    icon: ion-android-person
  - predicate:
      propertyKeys: []
      dbLabel: murder
      knownValues: {}
    icon: "\u2694\uFE0F"
  - predicate:
      propertyKeys: []
      dbLabel: weapon
      knownValues: {}
    icon: "\uD83D\uDD2B"
  - predicate:
      propertyKeys: []
      dbLabels: []
      knownValues: {}
quickQueries: []
sampleQueries: []

Download Recipe

Scenario

Ingest the killfeed websocket output from Daybreak Games' MMOFPS "PlanetSide 2", invoking the getJsonLines procedure to lazily fill out unknown static data. Replace all instances of s:example with a service-id acquired from http://census.daybreakgames.com/#service-id

Sample Data

Death events will stream in from the websocket once the recipe is started. The events are JSON objects like the one below.

{
    "payload":{
        "attacker_character_id":"5428010618015189713",
        "attacker_fire_mode_id":"26103",
        "attacker_loadout_id":"15",
        "attacker_vehicle_id":"0",
        "attacker_weapon_id":"26003",
        "character_id":"5428168624838258657",
        "character_loadout_id":"6",
        "event_name":"Death",
        "is_headshot":"1",
        "timestamp":"1392056954",
        "vehicle_id":"0",
        "world_id":"1",
        "zone_id":"2"
    },
    "service":"event",
    "type":"serviceMessage"
}

How it Works

The recipe connects an ingest stream to the PS2 Event Streaming WebSocket to manifest Death events as murder, victim, attacker, weapon, and character nodes in Quine.

INGEST-1 processes events emitted form the websocket:

YAMLJSON

- type: WebsocketSimpleStartupIngest
  url: wss://push.planetside2.com/streaming?environment=ps2&service-id=s:example
  initMessages:
  - |- 
    {
      "service":"event",
      "action":"subscribe",
      "worlds": ["all"],
      "characters":["all"],
      "eventNames":["Death"]
    }
  format:
    type: CypherJson
    query: |-
      WITH * WHERE $that.type = 'serviceMessage'
      CREATE (m:murder)
      SET m = COALESCE($that.payload, {})
      WITH id(m) as mId
      MATCH (murder) WHERE id(murder) = mId
      MATCH (victim) WHERE id(victim) = idFrom('character', murder.character_id)
      MATCH (attacker) WHERE id(attacker) = idFrom('character', murder.attacker_character_id)
      MATCH (weapon) WHERE id(weapon) = idFrom('weapon', murder.attacker_weapon_id)
      SET weapon.uninitialized = weapon.weapon_id IS NULL
      SET victim:character, attacker:character, weapon:weapon,
        victim.character_id = murder.character_id, attacker.character_id = murder.attacker_character_id,
        weapon.weapon_id = murder.attacker_weapon_id
      CREATE (victim)<-[:victim]-(murder)-[:attacker]->(attacker), (murder)-[:weapon]->(weapon)
      WITH murder, victim, attacker
      UNWIND [victim, attacker] AS character
      SET character.last_update = murder.timestamp

POST /api/v1/ingest/INGEST-1

{
  "type": "WebsocketSimpleStartupIngest",
  "url": "wss://push.planetside2.com/streaming?environment=ps2&service-id=s:example",
  "initMessages": [
    "{\n  \"service\":\"event\",\n  \"action\":\"subscribe\",\n  \"worlds\": [\"all\"],\n  \"characters\":[\"all\"],\n  \"eventNames\":[\"Death\"]\n}"
  ],
  "format": {
    "type": "CypherJson",
    "query": "WITH * WHERE $that.type = 'serviceMessage'\nCREATE (m:murder)\nSET m = COALESCE($that.payload, {})\nWITH id(m) as mId\nMATCH (murder) WHERE id(murder) = mId\nMATCH (victim) WHERE id(victim) = idFrom('character', murder.character_id)\nMATCH (attacker) WHERE id(attacker) = idFrom('character', murder.attacker_character_id)\nMATCH (weapon) WHERE id(weapon) = idFrom('weapon', murder.attacker_weapon_id)\nSET weapon.uninitialized = weapon.weapon_id IS NULL\nSET victim:character, attacker:character, weapon:weapon,\n  victim.character_id = murder.character_id, attacker.character_id = murder.attacker_character_id,\n  weapon.weapon_id = murder.attacker_weapon_id\nCREATE (victim)<-[:victim]-(murder)-[:attacker]->(attacker), (murder)-[:weapon]->(weapon)\nWITH murder, victim, attacker\nUNWIND [victim, attacker] AS character\nSET character.last_update = murder.timestamp"
  }
}

A standing query matches each new character node to populate the character properties via the census API.

YAMLJSON

- pattern:
    type: Cypher
    query: MATCH (newCharacter:character) WHERE newCharacter.character_id IS NOT NULL RETURN DISTINCT id(newCharacter) AS id
  outputs:
    populate-fresh-character:
      type: CypherQuery
      query: |-
        MATCH (c)
        WHERE id(c) = $that.data.id
        CALL loadJsonLines("https://census.daybreakgames.com/s:example/get/ps2:v2/character/?character_id="+c.character_id) YIELD value
        SET c += COALESCE(value.character_list[0], {})

/api/v1/query/standing/STANDING-1

{
  "pattern": {
    "type": "Cypher",
    "query": "MATCH (newCharacter:character) WHERE newCharacter.character_id IS NOT NULL RETURN DISTINCT id(newCharacter) AS id"
  },
  "outputs": {
    "populate-fresh-character": {
      "type": "CypherQuery",
      "query": "MATCH (c)\nWHERE id(c) = $that.data.id\nCALL loadJsonLines(\"https://census.daybreakgames.com/s:example/get/ps2:v2/character/?character_id=\"+c.character_id) YIELD value\nSET c += COALESCE(value.character_list[0], {})"
    }
  }
}

The character_list object returned from the API is similar to the one below.

{"character_list":[{"character_id":"5428010618020694593","name":{"first":"Dreadnaut","first_lower":"dreadnaut"},"faction_id":"1","head_id":"1","title_id":"97","times":{"creation":"1353434436","creation_date":"2012-11-20 18:00:36.0","last_save":"1508990907","last_save_date":"2017-10-26 04:08:27.0","last_login":"1508985950","last_login_date":"2017-10-26 02:45:50.0","login_count":"971","minutes_played":"101148"},"certs":{"earned_points":"206533","gifted_points":"9358","spent_points":"205791","available_points":"10100","percent_to_next":"0.002999999962901"},"battle_rank":{"percent_to_next":"43","value":"102"},"profile_id":"21","daily_ribbon":{"count":"5","time":"1508911200","date":"2017-10-25 06:00:00.0"},"prestige_level":"0"}],"returned":1}

A second standing query matches each new weapon node to populate the weapon properties via the census API.

YAMLJSON

- pattern:
    type: Cypher
    query: MATCH (weapon:weapon) WHERE weapon.uninitialized = true AND weapon.weapon_id IS NOT NULL RETURN DISTINCT id(weapon) AS id
  outputs:
    populate-weapon:
      type: CypherQuery
      query: |-
        MATCH (weapon) WHERE id(weapon) = $that.data.id
        CALL loadJsonLines("https://census.daybreakgames.com/s:example/get/ps2:v2/item?item_id="+weapon.weapon_id+"&c:join=weapon_datasheet") YIELD value
        SET weapon += COALESCE(value.item_list[0], {})
        REMOVE weapon.uninitialized

/api/v1/query/standing/STANDING-2

{
  "pattern": {
    "type": "Cypher",
    "query": "MATCH (weapon:weapon) WHERE weapon.uninitialized = true AND weapon.weapon_id IS NOT NULL RETURN DISTINCT id(weapon) AS id"
  },
  "outputs": {
    "populate-weapon": {
      "type": "CypherQuery",
      "query": "MATCH (weapon) WHERE id(weapon) = $that.data.id\nCALL loadJsonLines(\"https://census.daybreakgames.com/s:example/get/ps2:v2/item?item_id=\"+weapon.weapon_id+\"&c:join=weapon_datasheet\") YIELD value\nSET weapon += COALESCE(value.item_list[0], {})\nREMOVE weapon.uninitialized"
    }
  }
}

The item_list object returned from the API is similar to the one below.

{"item_list":[{"item_id":"7","item_type_id":"26","item_category_id":"139","is_vehicle_weapon":"0","name":{"de":"Spawn -Leuchte","en":"Spawn Beacon","es":"Baliza de apariciones","fr":"Balise de réapparition","it":"Faro di rigenerazione","tr":"Animation Control Sign"},"description":{"de":"Eine Signalleuchte zum schnellen Absetzen von Truppmitgliedern in einen Bereich aus geringer Höhe.","en":"A signal-emitting beacon that allows squad members to drop pod into an area from low orbit.","es":"Una baliza que emite una señal y permite a los miembros del escuadrón aterrizar con cápsula dentro de una zona desde una órbita baja.","fr":"Une balise émettrice de signaux qui module de aux membres de l'es un déploiement à partir d'une orbite basse.","it":"Faro emettitore di segnali che permette ai membri della squadra di inserirsi a caldo nell'area.","tr":"It allows squad members to come from low orbit to instantaneous space is a scattering checkmark."},"faction_id":"0","max_stack_size":"1","image_set_id":"1568","image_id":"3056","image_path":"/files/ps2 /images/static/3056.png","is_default_attachment":"0"}],"returned":1}

Running the Recipe

Warning

Before running this recipe beyond a few seconds, you'll need to apply for a service ID from Daybreak Games in order to access their API. The service ID "s:example" is available for casual use--it is throttled to 10 requests per minute per client IP address.

Update a local copy of this recipe once you receive your personal service-id.

Please don't share the updated recipe or your service ID with others.

❯ java -jar quine-1.6.2.jar -r planetside-2.yaml
Graph is ready
Running Recipe: Planetside 2
Using 4 node appearances
Running Standing Query STANDING-1
Running Standing Query STANDING-2
Running Ingest Stream INGEST-1
Quine web server available at http://localhost:8080

Once the recipe is running, you can explore the graph to find subgraphs formed around Death events.