Skip to main content
All CollectionsStepsSource Steps
HTTP Service Source Step – Paging Options
HTTP Service Source Step – Paging Options

Paging Options for the HTTP Service Source Step.

Praise Magidi avatar
Written by Praise Magidi
Updated over a week ago

The concept of an HTTP (Web) service is an open standard (JSON, XML, SOAP, HTTP, etc.) based web application that communicates with other web applications for the purpose of exchanging data. In practice, using a protocol such as HTTP to transfer data in machine-readable file formats such as XML and JSON is a standardized way of providing interoperability between disparate applications.

In Synatic, the HTTP Service Step provides this function and generally allows the flow to send requests to read or get data from a web-based API.

Illustrated below is a flow where the HTTP Service step is used as a Source step. A JSON step converts the data to JSON format. The Buffer destination step stores the data.

The paging option is used to set up how the data is rendered and controlled as retrieved from a web request. The Paging Option is shown in the image below. Click on the Enable checkbox as shown to access the paging options.

Delay – Time to wait (in milliseconds) before fetching the next page.

Page Size – Set the number of lines to retrieve for each page. In the example above, there will be 5 for each page.

Page Start – Set the number of lines to retrieve for each page.

Paging Options

The Paging section has some options that can be used when paging is enabled. Click on an option to open its configuration. The following options are available in the paging section.

Check Length

Set the maximum number of pages received to consider as complete. It is recommended to set a higher number just in case there are a lot of records that can exceed the estimated number of pages.

Check HTTP Status Code

After each page is retrieved, check for a specific HTTP Status Code to indicate the completion of the request. Paging will stop when the set status code is identified.

For more information on HTTP Status Codes, please click the link below:

The image below shows the output from the Check HTTP Status Code Paging option.

Check Field

Enter a field name that should exist on the return value.

This option is only applicable to the JSON format.

Field Path – The path to the field.

Additional Field – Additional fields to load.

Path separator – The path separator to use. Click on one of the options to select.

The image below shows the output from the Check Field Paging option.

Check Empty Array

Check if a field exists on the return value. This is only applicable to JSON formats.

Empty JSON Array – Check if the response is an empty JSON array when selected.

Field URL

Check specified field on the response to get the next URL from. In this example, each page is set to have 100 records in the Page Size, meaning that the next URL will start paging at 101.

Path separator – The path separator to use. Click on one of the options to select.

The image below shows the results from the flow where the Next URL starts paging at 101.

Did this answer your question?