Standing Query Wiretap¶

The standing query wiretap enables observing a standing query's results in real time over a WebSocket connection. A standing query wiretap can observe any of the following three stages of a standing query result: before processing begins, after transformation, or after full enrichment.

This is a diagnostic and visualization tool. The wiretap has no delivery guarantees. Results are dropped rather than buffered when your client connection is slow or the cluster is overloaded. The stream stops as soon as you close the connection.

For background on standing queries and how outputs and enrichment stages work, see Standing Queries.

Tap endpoints¶

All three endpoints are GET WebSocket connections. Use wss:// instead of ws:// if your instance is served over TLS (HTTPS).

Raw tap¶

Streams every StandingQueryResult as soon as the match is found, before any output workflow runs.

ws://{host}/api/v2/graph/{graphName}/standingQueries/{sqName}:tap

Use when you want to see raw match events before any enrichment is applied.

Pre-enrichment tap¶

Streams results after the preEnrichmentTransformation stage but before the Cypher resultEnrichment query runs.

ws://{host}/api/v2/graph/{graphName}/standingQueries/{sqName}/outputs/{outputName}:tap_pre_enrichment

Use when you have a transformation configured and want to inspect the result after preprocessing (such as metadata stripping) but before any enrichment is applied.

Post-enrichment tap¶

Streams the processed data that your configured destinations receive after all workflow stages have run.

ws://{host}/api/v2/graph/{graphName}/standingQueries/{sqName}/outputs/{outputName}:tap

Use when you want to verify exactly what is being sent to your output destinations without having to set up a separate destination to inspect it.

Message format¶

Each WebSocket message is a JSON string. The shape depends on the tap stage:

Raw tap: a StandingQueryResult object with a metadata field and a data containing the data directly returned by your standing query.
Pre-enrichment tap: the result after your preEnrichmentTransformation has been applied
Post-enrichment tap: the fully-enriched result object, identical to what destinations receive

Example uses¶

These examples require a running Quine instance with a standing query that is actively producing matches. A simple example would be a number iterator ingest limited to 1 per second, and a simple query that just labels a node.

{
  "name": "my-ingest",
  "source": {"type": "NumberIterator"},
  "query": "MATCH (n) WHERE id(n) = idFrom($that) SET n.num = $that, n : Number",
  "parameter": "that",
  "parallelism": 1,
  "maxPerSecond": 1
}

To be able to utilize the output taps, you should also add a destination with an enrichment query. Here is an example standing query you could use alongside the number iterator ingest above.

{
  "name": "numbers-sq",
  "pattern": {
    "type": "Cypher",
    "query": "MATCH (n : Number) RETURN strId(n) AS id",
    "mode": "MULTIPLE_VALUES"
  },
  "outputs": [
    {
      "name": "myoutput",
      "preEnrichmentTransformation":{ 
        "type": "InlineData"
      },
      "resultEnrichment": {
        "query":"MATCH (n) WHERE id(n) = $that.id RETURN (n.num) as num"
      },
      "destinations": [
        {
          "type": "Drop"
        }
      ]
    }
  ]
}

Command line standing wire tap¶

You can connect to the wiretap from any tool that supports websockets. For this example, we will use websocat. If you are using a HTTPS connection you will need to use wss instead of ws.

websocat ws://{host}/api/v2/graph/quine/standingQueries/numbers-sq:tap

This will provide a list of outputs that look something like:

{"meta":{"isPositiveMatch":true},"data":{"id":"0a362b50-12ff-3731-8d74-8f433a3b2338"}}
{"meta":{"isPositiveMatch":true},"data":{"id":"598319c2-9054-302b-8d0e-b9296c738ec4"}}
{"meta":{"isPositiveMatch":true},"data":{"id":"598319c2-9054-302b-8d0e-b9296c738ec4"}}

If you do not want the meta field and just want the id, you can tap into an output of a standing query that has preEnrichmentTransformation set to InlineData (like our example above). To do so, tap the pre-enrichment endpoint:

websocat ws://{host}/api/v2/graph/quine/standingQueries/numbers-sq/outputs/myoutput:tap_pre_enrichment

You will see results that look like this:

{"id":"0a362b50-12ff-3731-8d74-8f433a3b2338"}
{"id":"598319c2-9054-302b-8d0e-b9296c738ec4"}
{"id":"598319c2-9054-302b-8d0e-b9296c738ec4"}

To inspect the fully enriched result, tap the post-enrichment endpoint:

websocat ws://{host}/api/v2/graph/quine/standingQueries/numbers-sq/outputs/myoutput:tap

For our standing query enrichment query, the results will look like:

{"num": 1 }
{"num": 2 }
{"num": 3 }

Bookmarklets¶

A bookmarklet is a browser bookmark that runs JavaScript instead of opening a URL. To use one, create a new bookmark, paste the code as the URL, then click it while you have the Quine Exploration UI open.

The bookmarklet below connects to the raw tap for numbers-sq and automatically runs a graph query in the Exploration UI whenever a new match arrives.

javascript: (() => {
  const runQuery = (query) => {
    const queryBox = document.querySelector(".query-input-input");
    const nativeInputValueSetter = Object.getOwnPropertyDescriptor(
      window.HTMLInputElement.prototype,
      "value"
    ).set;
    nativeInputValueSetter.call(queryBox, query);
    queryBox.dispatchEvent(new Event("input", { bubbles: true }));
    setTimeout(() => queryBox.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true })), 0);
  };
  const ws = new WebSocket(
    "ws://{host}/api/v2/graph/quine/standingQueries/numbers-sq:tap"
  );
  ws.onmessage = (event) => {
    const e = JSON.parse(event.data);
    if (e.meta.isPositiveMatch) {
      runQuery('MATCH (n) WHERE strId(n) = "' + e.data.id + '" RETURN n');
    }
  };
})();

Replace {host} with your Quine host (e.g. localhost:8080), and use wss:// instead of ws:// if your instance is served over HTTPS.

How it works:

new WebSocket(...) opens a persistent connection to the raw tap for numbers-sq. Each time the standing query finds a new match, the server sends a message and ws.onmessage fires.

Each message has a meta field (with isPositiveMatch) and a data field. The data field contains one key for each column your standing query returns — in our case, id — because the pattern query uses RETURN strId(n) AS id. If the query instead returned RETURN strId(n) AS identifier, n AS number, the data fields would be e.data.identifier and e.data.number.

runQuery takes a query and submits it to the Exploration UI exactly as if you had typed it into the query bar and clicked the run button. The matched node appears in the graph after the query completes.