Skip to content

Reactive Streams

Reactive Streams enable TCP-based, backpressured communication between thatDot products.

Role Description Use Case
Publisher (Server) Binds to a port and broadcasts data to connected subscribers Standing query outputs
Subscriber (Client) Connects to a publisher to receive streamed data Ingest from another product's output

ReactiveStream as Ingest Source

The ReactiveStream ingest source connects as a client to an existing reactive stream publisher. Use this to ingest data from another product's standing query output.

Example

In this example, we configure Quine to ingest data from a reactive stream server running on port 9002.

Registering via the API

To register a ReactiveStream ingest via Create Ingest Stream: POST /api/v2/graph/quine/ingests:

{
  "name": "from-reactive-stream",
  "source": {
    "type": "ReactiveStream",
    "url": "localhost",
    "port": 9002,
    "format": {
      "type": "Json"
    }
  },
  "query": "CREATE (n:Event $that)",
  "parameter": "that",
  "parallelism": 1
}

Using a Recipe

The same ingest stream defined in a Recipe:

ingestStreams:
  - name: reactive-stream-ingest
    source:
      type: ReactiveStream
      url: localhost
      port: 9002
      format:
        type: Json
    query: |-
      MATCH (n)
      WHERE id(n) = idFrom($that.id)
      SET n = $that

Ingest Source Configuration Reference

Parameter Type Required Description
type string Yes Must be "ReactiveStream"
url string Yes Hostname of the reactive stream server to connect to
port integer Yes Port of the reactive stream server
format object Yes Record format, for example {"type": "Json"}. See Record Formats for the full list and the difference between Json and Raw.

ReactiveStream as Output Destination

The ReactiveStream output destination creates a server that broadcasts results to connected subscribers. Use this to publish standing query results for downstream processing by other products.

Example

Configure a standing query to output results to a reactive stream on port 9001:

{
  "name": "events-to-downstream",
  "pattern": {
    "type": "Cypher",
    "query": "MATCH (n:Event) RETURN DISTINCT id(n) AS id",
    "mode": "DISTINCT_ID"
  },
  "outputs": [
    {
      "name": "to-reactive-stream",
      "destinations": [
        {
          "type": "ReactiveStream",
          "address": "0.0.0.0",
          "port": 9001,
          "format": {
            "type": "JSON"
          }
        }
      ]
    }
  ]
}

Output Destination Configuration Reference

Parameter Type Required Default Description
type string Yes — Must be "ReactiveStream"
address string No localhost Address to bind the reactive stream server
port integer Yes — Port to bind the reactive stream server
format object Yes — Output format (JSON or Protobuf)

Cluster Limitation

Reactive Stream outputs do not function correctly when running in a cluster. Use Kafka or Kinesis for clustered deployments.


Troubleshooting

Connection Refused

If you see connection errors when starting a ReactiveStream ingest:

  • Verify the publisher (server) is running and bound to the expected port
  • Check that the url and port in your ingest configuration match the publisher's address and port
  • Ensure no firewall rules are blocking the connection

No Data Received

If the connection succeeds but no data flows:

  • Verify the format types match between publisher and subscriber (e.g., both using Json)
  • Check that the standing query output is actively producing results
  • Use the standing query statistics API to verify results are being generated

Backpressure

If data processing slows or stalls:

  • The subscriber may be overwhelmed; increase parallelism or optimize your ingest query
  • Check downstream systems (persistor, standing queries) for bottlenecks
  • Monitor Quine metrics for queue depth and processing rates

Additional Resources