The Group step can be used to group records with unique keys and compile the output into separate item arrays. This is useful when you want to extract, and group specified data from a data set and then process them later in your flow.
In this tutorial, we will use the Group step to extract data from a dataset retrieved from an HTTP Service.
For example, we'll use Todo list test data at https://jsonplaceholder.typicode.com/todos and group the Todo list items that have been completed and designated as True and those that have not been completed and designated as False.
There are several steps to creating, configuring, and using a Group step:
Watch the video below to learn how to add the Synatic Group step, configure and then test it.
Creating a flow to Group records
We'll create a flow, which is constructed as shown below, to illustrate the operation of the Group step.
The flow consists of associated steps to process and group a dataset. Click on the links below to learn more about adding and configuring each step.
Step 1 | Step 2 | Step 3 | Step 4 | Step 5 |
Adding the Group Step
The Group Step is located in the Mapper drawer in the Steps palette.
To use the Group Step in your flow:
Select and drag it onto the desired location in the flow after the Sort step.
Now you can configure it.
Configuring the Group Step
The Group step only requires you to define one or more key ID fields to group the data. In this tutorial, we'll only use a single field, namely 'completed', to uniquely identify records in the dataset.
When the Group step opens:
Click ⨁ to add a field.
Enter the key ID field you want to use to group the data.
In this case, we'll use the field, 'completed'.
Click the green ✔ to save.
When done, click Save Step.
Example Synatic Flow steps
The steps below describe the function of each of the individual steps required for the Flow to operate.
1. Retrieving the data
To retrieve data from an HTTP Service, we'll add an HTTP Service step as the first step and configure it to read from a specific web-based dataset.
2. Reading the data
The data from the source contains JSON data, so we'll add the JSON Reader to parse the data from the source.
To test the JSON reader step:
Select the blue play ▶ icon adjacent to the JSON step in the flow.
We can view the data from the source and get an idea of its format.
For clarity, each record retrieved has the following formatted structure:
{
"userId": 1,
"id": 1,
"title": "delectus aut autem",
"completed": "false"
}
The field we will use to uniquely identify each record has the field name 'completed'.
3. Sorting the records
To use the Group functionality, Synatic must first Sort the records. The Group is designed to keep memory-use efficient, so we'll add the Sort step to sort the data on the same defined key field.
When the Sort step opens:
Click ⨁ to add a field.
Enter the Order by key you want to use to order the data.
In this case, we'll use the same field, 'completed'.
Select the order of Ascending or Descending.
Click the green ✔ to save.
When done, click Save Step.
4. Grouping the records
If you examine the records in step 2, you will notice that some records have the Completed field value as “False”, and others have the Completed field value as "True”. The Group step will group all records with the same Completed value together.
For example, all the True records will be grouped together, and all the False records will be grouped together.
5. Displaying the Records
To test the flow display of the resulting data grouping up to this point, we'll add the Calculator step using just the default JavaScript code. The Calculator step is not typically needed, but we use it for this tutorial to display the grouped data when using the debug testing in the flow.
Now, let's test the flow after the Calculator step.
Select the blue play ▶ icon adjacent to the Calculator step in the flow.
We can see the two data arrays created by the Group step and get an idea of the data and its format.
Testing the entire Group flow
It is always advisable to fully test the Flow after completion to confirm an error-free flow operation.
1. Click the Execute Flow option above the flow canvas to initiate the test.
2. Click the Create Run option to start the test.
3. The test run may take a while, depending on the size of the dataset.
4. When the test run is complete, each step in the flow indicates the number of records processed, e.g., C: 200.
If any errors are encountered, it will indicate an E: xxx.
Any other line items, such as headers or footers, the run will indicate an O: xxx.
5. Click on the Test Records Completed button to display the list of arrays that were created.
6. Double-click on the first-row identifier to list all the records grouped under the False key.
7. Double-click on the second-row identifier to list all the records grouped under the True key.
This confirms that the Group step function was successful. You can now add functionality in the flow to process the arrays further.
Also read the Group reference article