Migrating from v1 Recipes¶
A v2 recipe uses v2 API entities instead of v1 API entities. The migration involves three kinds of changes:
- Structural changes — ingest streams and standing query outputs are reorganized (names are required, config is nested differently).
- Type renames — ingest source and destination type discriminators are renamed (e.g.,
PrintToStandardOut→StandardOut). - Enum value changes — enum values change from PascalCase to SCREAMING_SNAKE_CASE (e.g.,
DistinctId→DISTINCT_ID). See Enum Wire Format for the convention behind this change. - Change the
versionattribute from1to2.
The top-level recipe attributes (version, title, contributor, summary, description, ingestStreams, standingQueries, nodeAppearances, quickQueries, sampleQueries, statusQuery) remain the same.
Example¶
The following demonstrates migrating the Data Enrichment with Webhooks recipe from v1 to v2. The API v1 entities must be replaced with API v2 entities. For example, the ingestStreams attribute is an array of Ingest Stream Configuration objects in v1, but an array of Quine Ingest Configuration objects in v2.
version: 1
title: Data Enrichment with Webhooks
contributor: https://github.com/mastapegs
summary: Stream numbers into graph and notify HTTP endpoint to enrich graph
description: |-
This recipe will stream numbers into the graph and stream them out to an HTTP endpoint, which will
then calculate the factors of those numbers, and create relationships between the numbers and their
factors.
ingestStreams:
- type: NumberIteratorIngest
startAtOffset: 1
ingestLimit: 13
format:
type: CypherLine
query: |-
WITH toInteger($that) AS number
MATCH (n) WHERE id(n) = idFrom("Number", number)
SET n:Number, n.number = number
standingQueries:
- pattern:
type: Cypher
mode: DistinctId
query: |-
MATCH (n:Number)
WHERE n.number IS NOT NULL
RETURN DISTINCT id(n) AS id
outputs:
log-to-console:
type: CypherQuery
query: |-
MATCH (n:Number)
WHERE id(n) = $that.data.id
RETURN n.number AS number, $that.data.id AS id
andThen:
type: PrintToStandardOut
post-to-webhook:
type: CypherQuery
query: |-
MATCH (n:Number)
WHERE id(n) = $that.data.id
RETURN n.number AS number, $that.data.id AS id
andThen:
type: PostToEndpoint
url: http://127.0.0.1:3000/webhook
nodeAppearances:
- predicate:
propertyKeys: []
knownValues: {}
dbLabel: Number
label:
type: Property
key: number
prefix: "Number: "
quickQueries: []
sampleQueries:
- name: Return all Number nodes
query: MATCH (n:Number) RETURN n
statusQuery: null
version: 2
title: Data Enrichment with Webhooks
contributor: https://github.com/mastapegs
summary: Stream numbers into graph and notify HTTP endpoint to enrich graph
description: |-
This recipe will stream numbers into the graph and stream them out to an HTTP endpoint, which will
then calculate the factors of those numbers, and create relationships between the numbers and their
factors.
ingestStreams:
- name: number-iterator # ingest stream must have a name
source:
type: NumberIterator # type moves under source
startOffset: 1 # startAtOffset → startOffset
limit: 13 # ingestLimit → limit
query: |- # query moves out of format
WITH toInteger($that) AS number
MATCH (n) WHERE id(n) = idFrom("Number", number)
SET n:Number, n.number = number
standingQueries:
- name: number-processor # standing query must have a name
pattern:
type: Cypher
mode: DISTINCT_ID
query: |-
MATCH (n:Number)
WHERE n.number IS NOT NULL
RETURN DISTINCT id(n) AS id
outputs: # outputs changes from a map to an array
- name: log-to-console # output name moves to the `name` field
preEnrichmentTransformation:
type: InlineData
resultEnrichment:
query: |- # query moves under resultEnrichment
MATCH (n:Number)
WHERE id(n) = $that.id
RETURN n.number AS number, $that.id AS id
parameter: that
destinations: # andThen → destinations
- type: StandardOut # PrintToStandardOut → StandardOut
- name: post-to-webhook
preEnrichmentTransformation:
type: InlineData
resultEnrichment:
query: |-
MATCH (n:Number)
WHERE id(n) = $that.id
RETURN n.number AS number, $that.id AS id
parameter: that
destinations:
- type: HttpEndpoint # PostToEndpoint → HttpEndpoint
url: http://127.0.0.1:3000/webhook
nodeAppearances:
- predicate:
propertyKeys: []
knownValues: {}
dbLabel: Number
label:
type: Property
key: number
prefix: "Number: "
quickQueries: []
sampleQueries:
- name: Return all Number nodes
query: MATCH (n:Number) RETURN n
Structural Changes¶
Ingest Streams¶
In v1, each ingest stream is a flat object with type and format (which contains the Cypher query). In v2, ingest streams require a name, nest source config under source, and move the Cypher query to the top level.
| v1 | v2 |
|---|---|
No name (assigned INGEST-#) |
Required name field |
Top-level type |
source.type |
format: { type: CypherJson, query: ... } |
Top-level query; optional fileFormat or streamingFormat |
startAtOffset |
source.startOffset |
ingestLimit |
source.limit |
Standing Query Outputs¶
In v1, outputs is a map of name → output config. In v2, outputs is an array of objects with a name field, and the output pipeline is restructured.
| v1 | v2 |
|---|---|
outputs: { name: { ... } } (map) |
outputs: [ { name: "...", ... } ] (array) |
Top-level query in output |
resultEnrichment: { query: ..., parameter: ... } |
andThen: { type: ... } |
destinations: [ { type: ... } ] |
$that.data.id |
$that.id |
| N/A | preEnrichmentTransformation: { type: InlineData } |
Type and Enum Value Changes¶
Recipe v2 uses API v2 entities, which follow different naming conventions than v1. For the general convention, see Enum Wire Format.
Enum Values¶
Enum values change from PascalCase to SCREAMING_SNAKE_CASE. The v2 API rejects v1-style values.
| Field | v1 | v2 |
|---|---|---|
Standing query mode |
DistinctId |
DISTINCT_ID |
Standing query mode |
MultipleValues |
MULTIPLE_VALUES |
Quick query sort |
Node |
NODE |
Quick query sort |
Text |
TEXT |
fileIngestMode |
Regular |
REGULAR |
fileIngestMode |
NamedPipe |
NAMED_PIPE |
Ingest Source Types¶
Type discriminators remain PascalCase but are renamed to shorter forms. The type moves from the top level to source.type.
v1 type |
v2 source.type |
|---|---|
FileIngest |
File |
KafkaIngest |
Kafka |
KinesisIngest |
Kinesis |
KinesisKCLIngest |
KinesisKCL |
NumberIteratorIngest |
NumberIterator |
S3Ingest |
S3 |
SQSIngest |
SQS |
ServerSentEventsIngest |
ServerSentEvent |
StandardInputIngest |
StdInput |
WebsocketSimpleStartupIngest |
WebsocketClient |
Destination Types¶
v1 andThen.type values map to v2 destinations[].type:
| v1 | v2 |
|---|---|
PrintToStandardOut |
StandardOut |
PostToEndpoint |
HttpEndpoint |
PostToSlack |
Slack |
WriteToFile |
File |
WriteToKafka |
Kafka |
WriteToKinesis |
Kinesis |
WriteToSNS |
SNS |
CypherQuery |
CypherQuery |
Drop |
Drop |