The concept of using a Buffer Source step in a flow is to retrieve data from a Buffer that was previously saved. The stored data can then be retrieved repeatedly in other flows without having to revisit the original source.
A Buffer can be likened to physical storage, be it memory or disk, and temporarily hold data while moving between processes. Typically, the data is stored in a buffer as it is retrieved from a source (i.e., a database or another web service) before it is sent to another flow or device.
Using a Buffer Step
To add a source step:
1. Click on the highlighted icon as shown in the below image.
2. The below page will appear. Select or search for Buffer.
Configuring the Buffer Step
Illustrated in the below image is the configuration that is available in the Buffer Source step.
Buffer ID
The Buffer ID is the source Buffer that is used to retrieve data from in the flow. Click the dropdown to select a Buffer. Alternatively, click on the plus button to create a buffer.
Filter section
Boolean logic to apply logical filter operations - Select an appropriate operator from AND, OR, NOT.
Rules to apply comparative operators to return records with specified data values – Select a data field, an operator, the value, and the data type in each line.
Click the Add Rule or Add Group buttons to add more rules.
Fields tags to only display specified columns - Type in the field and then click on the tick icon to add.
Order By - List the data in a specific order using the Order By operator.
Click the + icon to add key index fields using the MongoDB Extended JSON specification.
Limit - The total number of records to return from the query.
Skip - Omit the first n number of records as specified.