Recipe

Full Recipe

Shared by: thatDot

Abstract

Recipe

# Recipe schema version (currently only supported value is 1; 🎉)
version: 1

# Identifies the Recipe but is not necessarily unique or immutable
title: Template Recipe

# URL to social profile of the person or organization responsible for this Recipe
contributor: https://github.com/example-user

# Brief copy about this Recipe
summary: This is a valid Recipe that ingests and writes back a file

# Longer form copy about this Recipe
description: |-
  This Recipe description
  has multiple lines.

# Ingest Streams define how data is processed and transformed
ingestStreams:
  - type: FileIngest
    path: $in_file
    format:
      type: CypherLine
      query: |-
        MATCH (n)
        WHERE id(n) = idFrom($that)
        SET n.line = $that

# Standing Queries define how data is transformed and output.
standingQueries:
  - pattern:
      type: Cypher
      query: MATCH (n) RETURN DISTINCT id(n) AS id
    outputs:
      output-1:
        type: CypherQuery
        query: |-
          MATCH (n)
          WHERE id(n) = $that.data.id
          RETURN n.line
        andThen:
          type: WriteToFile
          path: $out_file

# Customize node appearance in web UI.
nodeAppearances: [ ]

# Add queries to node context menus in web UI
quickQueries: [ ]

# Customize sample queries listed in web UI
sampleQueries:
  - name: Nodes
    query: MATCH(n) RETURN n LIMIT 10
  - name: Count Nodes
    query: MATCH (n) RETURN count(n)

# Optional Cypher query to be executed and reported to the Recipe user
statusQuery:
  cypherQuery: MATCH (n) RETURN n LIMIT 10

Download Recipe

Scenario

In this scenario, ...

Sample Data

Download the sample data to the same directory where Quine will be run.

How it Works

The recipe reads observations from the two sample data files using ingest streams to manifest a graph in Quine. A separate ingest stream is configured to process each file, each containing Cypher that parses the observations, manifests nodes, and relates them to each other in the graph.

INGEST-1 processes the endpoints.json file:

YAMLJSON

  - type: FileIngest
    path: endpoint.json
    format:
    type: CypherJson
    query: >-
        MATCH (proc), (event), (object)
        WHERE id(proc) = idFrom($that.pid)
        AND id(event) = idFrom($that)
        AND id(object) = idFrom($that.object)

        SET proc.id = $that.pid,
            proc: Process,
            event.type = $that.event_type,
            event: EndpointEvent,
            event.time = $that.time,
            object.data = $that.object

        CREATE (proc)-[:EVENT]->(event)-[:EVENT]->(object)

POST /api/v1/ingest/INGEST-1

{
  "type": "FileIngest",
  "path": "endpoint.json",
  "format": {
    "type": "CypherJson",
    "query": "MATCH (proc), (event), (object) WHERE id(proc) = idFrom($that.pid) AND id(event) = idFrom($that) AND id(object) = idFrom($that.object) SET proc.id = $that.pid, proc: Process, event.type = $that.event_type, event: EndpointEvent, event.time = $that.time, object.data = $that.object CREATE (proc)-[:EVENT]->(event)-[:EVENT]->(object)"
  }
}

A standing query is configured to detect a WRITE->READ->SEND->DELETE pattern that is typical for this type of exflitration event.

YAMLJSON

- pattern:
    type: Cypher
    query: >-
        MATCH (e1)-[:EVENT]->(f)<-[:EVENT]-(e2), 
            (f)<-[:EVENT]-(e3)<-[:EVENT]-(p2)-[:EVENT]->(e4)
        WHERE e1.type = "WRITE"
        AND e2.type = "READ"
        AND e3.type = "DELETE"
        AND e4.type = "SEND"
        RETURN DISTINCT id(f) as fileId

/api/v1/query/standing/STANDING-1

[
    {
        "pattern": {
        "type": "Cypher",
        "query": "MATCH (e1)-[:EVENT]->(f)<-[:EVENT]-(e2), \n      (f)<-[:EVENT]-(e3)<-[:EVENT]-(p2)-[:EVENT]->(e4)\nWHERE e1.type = \"WRITE\"\n  AND e2.type = \"READ\"\n  AND e3.type = \"DELETE\"\n  AND e4.type = \"SEND\"\nRETURN DISTINCT id(f) as fileId"
        },
        "outputs": {
        "stolen-data": {
            "type": "CypherQuery",
            "query": "MATCH (p1)-[:EVENT]->(e1)-[:EVENT]->(f)<-[:EVENT]-(e2)<-[:EVENT]-(p2), \n      (f)<-[:EVENT]-(e3)<-[:EVENT]-(p2)-[:EVENT]->(e4)-[:EVENT]->(ip)\nWHERE id(f) = $that.data.fileId\n  AND e1.type = \"WRITE\"\n  AND e2.type = \"READ\"\n  AND e3.type = \"DELETE\"\n  AND e4.type = \"SEND\"\n  AND e1.time < e2.time\n  AND e2.time < e3.time\n  AND e2.time < e4.time\n\nCREATE (e1)-[:NEXT]->(e2)-[:NEXT]->(e4)-[:NEXT]->(e3)\nWITH e1, e2, e3, e4, p1, p2, f, ip, \"http://localhost:8080/#MATCH\" + text.urlencode(\" (e1),(e2),(e3),(e4),(p1),(p2),(f),(ip) WHERE id(p1)='\"+strId(p1)+\"' AND id(e1)='\"+strId(e1)+\"' AND id(f)='\"+strId(f)+\"' AND id(e2)='\"+strId(e2)+\"' AND id(p2)='\"+strId(p2)+\"' AND id(e3)='\"+strId(e3)+\"' AND id(e4)='\"+strId(e4)+\"' AND id(ip)='\"+strId(ip)+\"' RETURN e1, e2, e3, e4, p1, p2, f, ip\") as URL RETURN URL",
            "andThen": {
            "type": "PrintToStandardOut"
            }
        }
        }
    }
]

Once Quine detects the pattern, the event is sent to a standing query output for additional processing and action.

    outputs:
      stolen-data:
        type: CypherQuery
        query: >-
          MATCH (p1)-[:EVENT]->(e1)-[:EVENT]->(f)<-[:EVENT]-(e2)<-[:EVENT]-(p2), 
                (f)<-[:EVENT]-(e3)<-[:EVENT]-(p2)-[:EVENT]->(e4)-[:EVENT]->(ip)
          WHERE id(f) = $that.data.fileId
            AND e1.type = "WRITE"
            AND e2.type = "READ"
            AND e3.type = "DELETE"
            AND e4.type = "SEND"
            AND e1.time < e2.time
            AND e2.time < e3.time
            AND e2.time < e4.time

          CREATE (e1)-[:NEXT]->(e2)-[:NEXT]->(e4)-[:NEXT]->(e3)

          WITH e1, e2, e3, e4, p1, p2, f, ip, "http://localhost:8080/#MATCH" + text.urlencode(" (e1),(e2),(e3),(e4),(p1),(p2),(f),(ip) WHERE id(p1)='"+strId(p1)+"' AND id(e1)='"+strId(e1)+"' AND id(f)='"+strId(f)+"' AND id(e2)='"+strId(e2)+"' AND id(p2)='"+strId(p2)+"' AND id(e3)='"+strId(e3)+"' AND id(e4)='"+strId(e4)+"' AND id(ip)='"+strId(ip)+"' RETURN e1, e2, e3, e4, p1, p2, f, ip") as URL
          RETURN URL
        andThen:
          type: PrintToStandardOut

The result once the pattern is detected is to output a link to the console that an analyst can use to review the event further within Quine's Exploration UI.

result

Running the Recipe

❯ java -jar quine-1.8.2.jar -r .yaml

Summary

Summary.

Tip

Quick Queries are available by right clicking on a node.

Quick Query	Node Type	Description
Adjacent Nodes	All	Display the nodes that are adjacent to this node.
Refresh	All	Refresh the content stored in a node
Local Properties	All	Display the properties stored by the node

What is your call to action?