A log is kept of the records that are completed successfully, records that have an error, and records that were excluded or skipped due to filtering. Other dataset line items that represent headers and footers are also logged. All these logs are stored in a Record Event Buffer.
Illustrated in the image below is an example of a flow that has a Record Generator as a source step. The record information is stored in the Record Event Buffer.
Adding a Record Event Buffer Step
To add a Record Event Buffer step in a flow, follow the below instructions:
1. Click on the highlighted icon as shown in the below image.
2. The below page will appear. Click on the RecordEvent tab and select Buffer as shown in the image below.
Configuring a Record Event Buffer Step
Illustrated in the below image is the configuration that is available in the Record Event Buffer step.
Events – The events to trigger on. This field is required for the step to work. The following options are available:
Completed – Log all records that are successfully processed.
Error – Log all records that contain errors.
Skipped – Log all filtered and excluded records.
Header – Log all line items that are identified as a header within the dataset.
Footer – Log all line items that are identified as a footer within the dataset.
Buffer ID – Select an existing buffer by clicking on the dropdown arrow as shown in the image above. Alternatively, click on the highlighted plus button to create a new buffer.
Max – Set the maximum number of records to log. If more than one event is selected, please note that it will show the status of the records that were processed first.
Clean Record – Remove the record details from the log and retain only record headers.
Save Parameters – Save any parameters associated with the flow in the log.
Purge Before – Purge the buffer before the run starts.
Batch Size – Set the number of records to cache before submitting to the log.
The example in the Calculator step as shown in the image below will throw/return as an error.
The below payload shows the fields that get stored in the Record Event Buffer and the error message.
Troubleshooting/Debugging a Run
For more information on how to troubleshoot/debug a Run, please click the link below: