The concept of using a Source Buffer step in a flow is to retrieve data from a Buffer that was previously saved. The stored data can then be retrieved repeatedly in other flows without having to revisit the original source.
A Buffer can be likened to physical storage, be it memory or disk, and temporarily hold data while moving between processes. Typically, the data is stored in a buffer as it is retrieved from a source (i.e., a database or another web service) before it is sent to another flow or device.
In Synatic, you can retrieve data from a Buffer using a Source Buffer step.
Using a Buffer Step
The Buffer Step is located in the Source drawer in the Steps palette. To use the Buffer Step, select and drag it to the desired location in the flow, and use it to provide the source data to the flow.
Let's say we want to retrieve only customer data whose email address has a '.biz' top-level domain (TLD) in the Domain Name.
Configuring the Buffer Step
The configuration for this step has several options.
The Buffer ID is the source Buffer that is used to retrieve data from in the flow.
Click ⨁ to create a new Buffer, or
Click Select a Buffer to select the required Buffer in the selection list.
This tab enables you to build a filter query to define the data you want to retrieve for the Buffer.
The Query Builder uses the following criteria to set up the filter.
Boolean logic to apply logical filter operations.
• Select an appropriate operator from
AND, OR, NOT.
Rules to apply comparative operators to return records with specified data values.
• Select a data field, an operator, the value, and the data type in each line.
• To add more rules, click the Add Rule or Add Group buttons.
Field tags to only display specified columns.
• Click on Add a tag, then type in the column name.
Sort fields to list the data in a specific order using the Order By operator.
• Click ⨁ to add key index fields using the MongoDB Extended JSON specification.
Limit is used to set the total number of records from the Buffer.
Skip is used to omit the first
nnumber of records as specified.
• As an example, enter 5 into the input to omit the first five records.
The Raw Query option lets you examine the filter query with the comparative operators displayed in MongoDB Query Language.
Limitations and known issues
Read about other Source Steps
Revisit the Getting Started page for a refresher.
Read more about Flow Steps here.
Read more about Buffers here.