All Collections
Steps
Source Steps
HTTP Service Source Step
HTTP Service Source Step

An HTTP Service commonly provides communication with a web-based application server.

Praise Magidi avatar
Written by Praise Magidi
Updated over a week ago

The concept of an HTTP (Web) service is an open standard (JSON, XML, SOAP, HTTP, etc.) based web application that communicates with other web applications for the purpose of exchanging data.

In practice, using a protocol such as HTTP to transfer data in machine-readable file formats such as XML and JSON is a standardized way of providing interoperability between disparate applications.

In Synatic, the HTTP Service Step provides this function and generally allows the flow to send requests to read or get data from a web-based API.


Using the HTTP Service Step

The HTTP Service step is located in the Source drawer in the Steps palette. To use the HTTP Service step:

  1. Select and drag the HTTP Service step to the desired location in the flow.

  2. Give the step a name and a short meaningful description.

Note: A prerequisite before adding the HTTP Service step:

  1. First insert the required processing steps in the flow, for example, JSON followed by a Mapper, and then a CSV writer step.

  2. Next, insert the HTTP Service step in front of the JSON step.

  3. The processing steps can then process the response from the HTTP Source step.


Configuring the HTTP Service step

The configuration for this step involves several options and settings.

URL

  • Enter the URL of the HTTP Service.

HTTP Method

  • The two most common HTTP methods used are GET and POST.

  • Learn more about HTTP Methods.

Authentication Method

Click the + icon to select and configure the HTTP Authentication type:

  • Auth Type - None - No authentication is required.

  • Auth Type - Basic - Requires a standard username and password.

  • Auth Type - OAuth2 is a protocol for distributed authorization. Generally, OAuth2 provides clients a secure delegated access to server resources on behalf of a resource owner.
    - See more detail in the Example below.

  • Auth Type - Header


Options

Headers

  1. Click the + icon to add any Headers you want to submit to the HTTP Service.

  2. Enter the headers you want to submit.

Paging

  • The paging option lets you set up how the data is rendered and controlled as retrieved from a web request.

  • Enabled - Select the checkbox to activate the paging criteria options.
    - Page Size - Set the number of lines to retrieve for each page.
    -Page Start - Set the page to start the retrieval. This will usually be set to 1.

The next four options are used to test for the completion of the retrieval.

  • Check Length - Set the number of pages received to consider as complete.

  • Check HTTP Status Code - After each page is retrieved, check for a specific status code to indicate the completion of the request.

  • Check Field - Enter a field name that should exist on the return value.
    Note: This is only applicable to the JSON format.

  • Check Empty Array - Check for an empty array [ ] to indicate the completion of the request.
    Note: This is only applicable to the JSON format.

Keep Headers

  • Select the Keep Headers checkbox to save the HTTP response headers to the headers field in the processing object.

  • This option lets you use the metadata for any specific purpose.


Advanced Mode Options

Click on the Advanced Mode text button to display the following advanced options:

  • Follow Redirects - Select the checkbox to instruct the HTTP Service step to obey and follow any redirect it might encounter.

  • Relax SSL - Select the checkbox to instruct the HTTP Service step to accept self-signed certificates that are otherwise rejected.


Examples

Auth Type: oAuth2

When selecting an Auth Type of oAuth2 in a typical HTTP Service request to an API, the following type of configuration may apply:

https://authorization-server.com/oauth?
response_type={code}
&client_id={CLIENT_ID}
&redirect_uri={"https://admin.synatic.com/app/public/oauthresponse"}

Configure the following fields with the correct values to process successful web requests:

Before you can begin the OAuth process, you must first register a new app with the service. When registering a new app, you usually register basic information such as application name, website, logo, etc.

In addition, you must register a redirect URI to be used for redirecting users to a web server.

Note: The options marked with an * are required fields.

  • OAuth Provider URL* - The OAuth2 provider URL to call for authentication.

  • OAuth Token URL* - The OAuth2 provider URL to generate tokens from.

  • Client ID* - The client id issued by the provider.
    After registering your app, you will receive a client ID and optionally a client secret. The client ID is considered public information and is used to build login URLs, or included in Javascript source code on a page.

  • Client Secret* - The client secret issued by the provider.
    The client secret must be kept confidential. If a deployed app cannot keep the secret confidential, such as single-page Javascript apps or native apps, then the secret is not used, and ideally, the service shouldn't issue a secret to these types of apps in the first place.

  • Response Type* - Select the Response type that the request will respond with as either a Code or a Token. This indicates that your server expects to receive an authorization code

  • Grant Type* - Select the oAuth2 Grant type to use: Authorization Code for apps running on a web server, browser-based and mobile apps.

  • Use Refresh as Token - Select this option if your provider does not support refresh tokens but uses long-lived tokens.

  • One use Refresh Token - Select this option if your provider does not support using a refresh token more than once.

  • No Query String - Select this option if you don't want to send parameters in the query string.

  • Scopes - A Scope is a mechanism in OAuth 2.0 that limits an application's access to a user's account. An application can request one or more scopes, and the access token issued to the application will be limited to the scopes granted.

  • Additional QS Values - You can also add additional Query Strings (QS) to the HTTP Service request by adding parameters here.
    Note: These parameters will not be submitted If you select the No Query String checkbox.

Auth Type: Basic

When selecting an Auth Type of Basic in a typical HTTP Service request to an API, the following type of configuration may apply:

 auth=HTTPBasicAuth('user', 'pass'))

Enter your credentials as a standard username and password.


Limitations and known issues

None.


See Also

Read more about Source Steps here.


Useful Tutorials

Revisit the Getting Started page for a refresher.

Read more about Steps here.

Read other tutorials using the HTTP Service step:

Did this answer your question?