The Group step can be used to group records with unique keys and compile the output into separate item arrays. This is useful when you want to extract, and group specified data from a data set and then process them later in your flow.

In this tutorial, we will use the Group step to extract data from a dataset retrieved from an HTTP Service.

For example, we'll use Todo list test data at https://jsonplaceholder.typicode.com/todos and group the Todo list items that have been completed and designated as True and those that have not been completed and designated as False.

There are several steps to creating, configuring, and using a Group step:

Creating a Synatic flow to Group records

Adding the Group Step

Configuring the Group Step

Synatic Flow steps

Testing the Group flow


Watch the video below to learn how to add the Synatic Group step, configure and then test it.


Creating a flow to Group records

We'll create a flow, which is constructed as shown below, to illustrate the operation of the Group step.

Synatic - Insert Group Step

The flow consists of associated steps to process and group a dataset. Click on the links below to learn more about adding and configuring each step.

Step 1

Step 2

Step 3

Step 4

Step 5

HTTP Service

JSON

Sort

Group

Calculator


Adding the Group Step

The Group Step is located in the Mapper drawer in the Steps palette.

To use the Group Step in your flow:

  1. Select and drag it onto the desired location in the flow after the Sort step.

  2. Now you can configure it.


Configuring the Group Step

The Group step only requires you to define one or more key ID fields to group the data. In this tutorial, we'll only use a single field, namely 'completed', to uniquely identify records in the dataset.

Synatic - Edit Group Step

When the Group step opens:

  1. Click to add a field.

  2. Enter the key ID field you want to use to group the data.

  3. In this case, we'll use the field, 'completed'.

  4. Click the green ✔ to save.

  5. When done, click Save Step.


Example Synatic Flow steps

The steps below describe the function of each of the individual steps required for the Flow to operate.


1. Retrieving the data

To retrieve data from an HTTP Service, we'll add an HTTP Service step as the first step and configure it to read from a specific web-based dataset.


2. Reading the data

The data from the source contains JSON data, so we'll add the JSON Reader to parse the data from the source.

To test the JSON reader step:

  • Select the blue play ▶ icon adjacent to the JSON step in the flow.

  • We can view the data from the source and get an idea of its format.

Synatic - Display JSON debug data for the Group step

For clarity, each record retrieved has the following formatted structure:

{
"userId": 1,
"id": 1,
"title": "delectus aut autem",
"completed": "false"
}

The field we will use to uniquely identify each record has the field name 'completed'.


3. Sorting the records

To use the Group functionality, Synatic must first Sort the records. The Group is designed to keep memory-use efficient, so we'll add the Sort step to sort the data on the same defined key field.

Synatic - Edit the Sort step

When the Sort step opens:

  1. Click to add a field.

  2. Enter the Order by key you want to use to order the data.

  3. In this case, we'll use the same field, 'completed'.

  4. Select the order of Ascending or Descending.

  5. Click the green ✔ to save.

  6. When done, click Save Step.


4. Grouping the records

If you examine the records in step 2, you will notice that some records have the Completed field value as False”, and others have the Completed field value as "True. The Group step will group all records with the same Completed value together.

For example, all the True records will be grouped together, and all the False records will be grouped together.

Synatic - Edit the Group step

5. Displaying the Records

To test the flow display of the resulting data grouping up to this point, we'll add the Calculator step using just the default JavaScript code. The Calculator step is not typically needed, but we use it for this tutorial to display the grouped data when using the debug testing in the flow.

Synatic - Edit the Calculator step

Now, let's test the flow after the Calculator step.

  • Select the blue play ▶ icon adjacent to the Calculator step in the flow.

  • We can see the two data arrays created by the Group step and get an idea of the data and its format.

Synatic - Display JSON debug data for the Calculator step

Testing the entire Group flow

It is always advisable to fully test the Flow after completion to confirm an error-free flow operation.

1. Click the Execute Flow option above the flow canvas to initiate the test.

Synatic - Execute Run on Group flow

2. Click the Create Run option to start the test.

Synatic - Create Run option

3. The test run may take a while, depending on the size of the dataset.

Synatic - Test run in progress

4. When the test run is complete, each step in the flow indicates the number of records processed, e.g., C: 200.

  • If any errors are encountered, it will indicate an E: xxx.

  • Any other line items, such as headers or footers, the run will indicate an O: xxx.

Synatic - Test run complete

5. Click on the Test Records Completed button to display the list of arrays that were created.

Synatic - Display records processed during test run

6. Double-click on the first-row identifier to list all the records grouped under the False key.

Synatic - Display test run records of first group

7. Double-click on the second-row identifier to list all the records grouped under the True key.

Synatic - Display test run records of second  group

This confirms that the Group step function was successful. You can now add functionality in the flow to process the arrays further.


Also read the Group reference article

Did this answer your question?